Boooooom, John with an epic Friday afternoon answer!

Thanks John, that deserves a beer...

In general yes, it's much more performant to serve images separately, particularly for static content and high volume image sites (flickr etc).

Browsers supporting data:image/jpg;base64 was not designed with JSON in mind, I think it was more to do with embedding images in an HTML page to reduce page load speed (reduce the amount of requests), where for instance there are lots of small icon images.

Reducing requests might not be a strong reason to combine images, but the fact that this is an established practice does cancel out some arguments against one slightly large request vs two normal sized requests.

So if these types of reasons are not important, the remaining arguments are less why not, but more why would you want to.

One good reason is if you want to move binary data and meta data in one single message, particularly if you require transactional integrity. This can be achieved by sending the meta data and binary data in separate multi parts over HTTP, or the binary data can be embedded inside the meta data. This latter practice has been around for some time where images and documents are embedded inside XML over SOAP.

The continued popularity of JSON for web API's means that more and more developers are looking at the same problem of moving meta data and binary data around in a single message / transaction, not just to browsers but between servers.

If you look at the FHIR specification, for instance, you will see it has a base64Binary type.

As you say, it may be preferable to make a separate request, but maybe only where performance is the strongest non-functional requirement. If this is not an issue, then there are few anti reasons to counter argument the benefits of a single message object.

Hi Ray,

The trouble is determining if a number value is also a string type or a special number type, as comparisons can give different answers for numbers starting with a zero...

USER>s x=0.12345
 
USER>w (x=+x)
1
USER>s x="0.12345"
 
USER>w (x=+x)
0

The obvious answer is to do (+x=+x) but this does not solve how to unit test the type and value.

I agree that on reflection the dependency on local collation would not work for my unit test framework as it would restrict its scope of use, but still an interesting answer.

Any more suggestions...

I figured out that $length can detect a stringy number starting with zero that is not dependent on local collation...

Is a string type...

USER>s x="0.12345"
 
USER>w $l(x)'=$l(+x)
1
 

Is not a string type...
USER>s x=0.12345
 
USER>w $l(x)'=$l(+x)
0

BUT,  this or "sort after" will only work for values starting with a zero.

I could use this to fix the specific generic assertion test failure I have, but it would be nice to expand the unit test methods to have an AssertNumberEquals().

It might be that I have to settle on...

>w ["1"].%GetTypeOf(0)
string

And only enable this method in supported versions.

Thanks Ray, but coercing the unit test to force a pass will cloak the underlying problem.

Let me expand on my original post, we know in COS that many variables start out as stringy values, no matter if they contain a number or not...

COS coercion operators do a pretty good job at dealing with numbers inside strings, except that there is an inconsistency in the equality operator when comparing floating point numbers. If "1.5"=1.5 is true, then arguably "0.5"=0.5 should also be true, but it is not. This means that developers should be wary of automatic equality coercion on floating point numbers.

To compound this problem, the COS compiler will ignore the following two potential problems...

1. Assigning a stringy value to a %Float property
2. Returning a stringy value from a method with return type of %Float

Which can lead to a false understanding of what a developer is dealing with.

To make things a little more interesting, a persistent object will automatically coerce a %Float property to a true number value when saved. That's fine, but what if the developer is unaware that he / she is assigning a stringy float value and later performs a dirty check between another stringy float value and the now saved true float number. The code could potentially be tripped up into processing some dirty object logic when nothing has changed.

As developers we need to code defensively around this problem, probably the best thing that we can do is always manually coerce a variable / property at source when we know it's a floating point value...

But, since we are not perfect, and the compiler won't help us, it's easy to consider that a few bugs might slip through the net.

This is where unit testing and good code coverage should highlight these exact types of problems. In this instance, a unit test should fail if the two values are not both the same type and value. So the implementation of AssertNumberEquals should check both type and value. Therefore, both of the following comparisons should fail...

"1"=1

"0.12345"=0.12345

This is why as I originally posted that (+"0.12345"=0.12345) is not the right answer, as it will create a false positive.

So the question boils back to, how do you detect the runtime type of a variable or property.

One solution that I have come up with so far would roughly look like...

Except that it is dependent on recent versions of Caché.

What I need is a similar solution that would be backwards compatible. 

Thanks Alexander, but the AssertNumberEquals method should create a failed unit test when the values are of different type. I have updated my question to be a little more clear on this. 

Hi Ray, thanks for the long responses, these will be great for anyone new to Caché.

No imposing coding convention here, just 20 years on the rock face with Caché/COS and a good pattern of the trip hazards inherent in the language, as all languages do (love COS, no bashing here).

I've been evolving a new version of a unit test framework I have been using for years and want to make sure that it handles some of these regular trip hazards.

In this instance, I have my own backwards compatible JSON library that failed a test because it was assigning a stringy number to a %Float property in its own normalisation method...

https://github.com/SeanConnelly/Cogs/blob/master/src/Cogs/Lib/Json/Cogs.Lib.Json.ClassDeserializer.cls

If I can add a new assert method as described earlier, I can catch this type of problem upstream and prevent potential bugs leaking out into live code.

So back to the simple question, would be great if anyone at Intersystem's knows any way to check the type of a variable, I can't think of anything from my legacy ANSII M days, perhaps there is a $zu function or similar?

Thanks Ray.

Btw, I found a solution earlier, I've added an answer to the post.

I accept that implementations like $lb might change in the future, but I now have a backwards compatible solution that can work along side the dynamic object solution on future releases. Comparing the two outputs in itself will make a good unit test of the unit tester on installation.

Agreed, this is a highly specialised use case, specifically for unit testing against potential floating point equality errors. Using IsString() as a day to day function would in most cases be a bad thing.

Just to clarify, the sorts after suggestion does NOT work, whilst it can detect stringy fractions, it does not work for even the simplest floating point number...

>w "1.1"]]$c(0)
0

> The difference is that the method above will fail numbers in canonical form

Do you have a specific condition where it will fail, I tested 1.6e+8 fractional number tests without any problem, so obviously concerned that there are conditions where it fails that I have not thought about yet.

On reflection I agree, unit testing simple return types is pointless.

It's only return objects with %Float properties that would need to be unit tested for type as well as value...

I want the float member to be a canonical number, not a string.

So a unit test would look like...

AssertNumberEquals(x.Amount,0.1)

which would fail, this would require a change in the method code to...

set test.Amount=+$p(data,",",2)

Which means the unit test will now pass, and quirky things won't happen downstream.

Thanks for the help Ray, it's been a really interesting conversation.

Hi Sergey,

> 1. No client caching

You will obviously know this already, but JSON can of course be cached (with its image) on the client if required.

However, images served in JSON are typically private images served via a secure API and should never be cached. As such, all caching benefits go out of the window anyway, whether your using JSON or not.

> 2. No lazy-loading

Fetching images dynamically in JSON does not stop it from being a lazy-loading technique, if anything it is perfectly suited to loading images just when they are needed.

> There are also other downsides possible.

There is really only one technical downside, images posted as Base64 are inflated by 37%. If you are paying for network bandwidth by the $ then you might think twice, even if web server compression will bring this back down.

Of course continue to serve static unsecure images as binary over HTTP, particluarly if you want to leverage on automatic image caching.

If you are building a rich API (REST/RPC) with a JSON payload, then don't be afraid of embedding images. This will make for consistent API's.

When we build SOAP interfaces, we don't expect a SOAP client to drop out of the SOAP protocol to raw HTTP to make a seperate request for an image. How would we describe this via a WSDL to the soap client? How should the authentication for this be implemented? Some kind of token that needs its own seperate emergency hatch in the server side API?

The same will eventually be true for REST and JSON-RPC. It doesn't matter if this is server to server, or client to server, what we really want is a consistent API that can be described and authenticated in a consistent way (e.g SWAGGER + OAuth 2.0).

Bottom line, to say there are "big problems" is a little FUD, and I hope will not deter other developers from considering this approach before they have done their own research.

Sean.

There are 101 repo's marked with intersystems-cache so we could probably just pass this test, even if some of them are using .xml exports.

Is this the ATOM language lib that you mentioned...

https://github.com/tsafin/atom-language-cos