I was very interested in this question! Looking at this really great post here, there was some documentation I wanted to look at but it's giving me access errors :( hoping to read up on the security package documentation soon!

Depending on how your column is set up, you may have a couple options. If what you're returning is pretty simple, you could embed javascript into the layout editor code view to access the dom of the other columns values you want to check. Another option which is easier to maintain and control, and is closer to what you mentioned is adding a transformation which calls out to an ObjectScript class function. From there you can access the context of whatever data you're pulling and check the values right there, and return whatever you want in your column.