All
Accepted answers
Peter Cooper · Jun 21, 2017 go to post

Hi Kyle

Thanks for your excellent comment

I agree - sort of....

Bit it's a balance (as always) between loading buffers - it may be the case that there is an occasional need to just grab the dates - but if it's only a 10% (say) need whereas the 90% need is to display/process the header and the lines together then, for me, the 90% need should win out.

Also if the dates (or whatever) are indexed then a selection on a date range (say) will only select the required rows from the index.

= =

But as I said before - it depends on the size of the system - my clients have modest needs (maybe 3 million rows max) and with a 64Gb machine all/most of the blocks are in memory anyway smiley

But thanks for the thoughts - I will certainly start looking at one-many with a cascade delete

Peter

Peter Cooper · Jun 24, 2017 go to post

Hi Wolf

Long time no meet !!!

This is such an interesting conversation...

And it all depends on on the actuality - if you have a parent with a sqillion children - then the answer to p/c performance is not so good

Also if you have a business object where the many side are constantly added to - eg a patients lab results- again different - leads to block splitting and a bunch of pointer blocks having to be loaded into memory

So...

my business case argument is with an Invoice header and invoice lines then parent/child v one/many is the most efficient - they (really never change) so no index bock splitting

But I do take Dan's and Otto's comment about bitmaps only doing integer ID's

= =

Tell you a story..

around 12 tears ago I was involved with a client with a 1.2Tb database - I was critisised by the in-house staff for IT staff for using naked global references in one module - blab blah - but it was 2% faster than full global references and  in a tight loop 2%  meant 30 mins off the processing time

= =

Having said that - I *will* be trying out one-many with a cascade delete

Peter

Peter Cooper · Jun 24, 2017 go to post

FAO Mike Kiddow

All of us Cache gurus are getting carried away with the internals and very deep design concepts that rely on an in-depth knowledge of how Cache works.....

I guess you do not fully follow what we are discussing - (sorry not being dismissive at all) - it's just that me, Wolf, Dan Otto, Kyle are just being super-dooper Cache egg heads.

Bottom line is...

That for any reasonable sized database (ie less than <~10,000,000) rows on a modern server with adequate memory - it don't much matter!!! Cache will do the biz.

= =
But please note that the example from Scott is not correct...
Having real children with real parents then the parent-child relationship is incorrect on many levels - because as I said before "a parent can die" and you don't want the children records to disappear!!!!

In case I would have (as a first pass) a class "people" and another class "people relationships" which links different instances of people with each other

= =

oo methodology seems intuitively easy to understand - but it's not
and on top of that is the pros and cons of Cache performance considerations that we have been discussing

But give it a go
As I said it don't much matter unless you are in serious terabyte country

Peter

Peter Cooper · Mar 13, 2018 go to post

Hi Evgeny

We have met briefly at last autumn's Developer meet at the Belfry - keep up the great work

New Tag - absolutely (please)
Angular2, Angular4 and now Angular5 are enhancements of the same basic product Angular1 (now AngularJS) is a different thing

Have a look at https://dzone.com/articles/learn-different-about-angular-1-angular-2-am…
(that's not a typo the URI is as spelt)

or google differences between angularjs and angular2

Peter

Peter Cooper · Mar 15, 2018 go to post

Hi Evgny

Thanks - Only hope I can find the time and energy to keep it up :}

Some questions about the community HTML editor....
1. Is there an easy way to add hyperlinks in-line or do you have to manually edit the source
2. Also are there any checking rules that are enforced re hyperlinks
3. How do you upload images (screen shots)
4. Is it possible to embed video (again screen shots) or is it better to use YouTube and add a link
5. Is there a better free HTML editor/method of working to use to create articles rather than  built in?

Peter

Peter Cooper · Mar 15, 2018 go to post

Hi All

The way I work is:- Personal Development Machines - Deploy to Test Server for User Acceptance Testing - Deploy to Live

The version control system that I use is TrakWarePro from Globalware - sadly no longer in existence  - but it works for us.  Not only to maintain versions but to ship releases between the three environments.

When deploying the classes need to be compiled (obviously) but I don't trust ISC compiling *all* the required classes and SQL statements etc.  Neither does $System.OBJ.CompileAll() work 100% in resolving the dependencies in the correct order.

Also a release will need to set SQL access on tables for any new data classes.

So I have developed a do ##class(setup.rCompileAll).doit() method that does all the necessary - in the correct order, set the SQL Access etc.

Usually a deployment will require changing data/updating indices/adding pages to the access database etc etc - so there is usually a setup class that contains the code to do this.

So I have

  • a Studio project "pfcXXX"
  • a version control project "pfcXXX"
  • a setup class "setup,rPFCxxx.cls"

And all this works 99.9% over 10 plus years - I can't actually remember wen it went wrong but nothing in this world is 100% smiley

The downtime can be as little as 5 minutes for a simple release or up to 1 hour or so if the data changes are complex.

The only downside is system is unavailable to the user whilst this process is happening - I know about the technique of using mirroring and updating one mirror and then swapping - but this is overkill for my users.

Peter

Peter Cooper · Mar 16, 2018 go to post

Hi Robert and all

You can achieve the separation quite easily with routine and class package mapping.

I have a client that has overseas affiliates they all share the same code/class base but have their own namespace with routines and packages mapped to the master namespace.

Works just fine

The only issue is that developing the code is more complex as the different affiliates have started to need different functionality starting from the same base screen

Peter

Peter Cooper · Mar 16, 2018 go to post

Just had to do this at a client 
And missed out another thing that I have to do....

This is a ZEN app that has custom components sub classed from ZEN - these create css and js files

But only in the master namespace/csp application

These files need to be copied to the other namespace/csp applications

peter

Peter Cooper · Mar 16, 2018 go to post

Hi John
Well hidden indeed - you would only find it by reading every line of the docs for each release

Peter

Peter Cooper · Mar 16, 2018 go to post

Just realised that copying the Cache.dat is only sensible for a single developer
If you have more than one developer working on different projects all deploying to the same Test server then copying won't work - you would get bits and pieces from different projects

Even for a single developer it's a bit dodgy - you could be working on two or more projects at the same time - waiting on user acceptance testing - if one passes then you want to deploy that to live but not the  others.

The more I think about it the more I believe that my method of working is the only one that works for all possibilities - or so I believe - if anyone has a better method please tell

Peter

Peter Cooper · Mar 16, 2018 go to post

Hi Evgeny

What situation do you have in mind that could cause the compilation to be unsuccessful?

With proper version control and release procedure this has rarely happened in my experience - when it does it's been due to unresolved dependencies - and in this case a re-compile fixes it.

There is one possibility where it *could* happen - that is if the VC system allows multiple reservations/branches for the same class - bet we don't allow that.

= =
I can't see how deploying/copying the Cache.dat will avoid problems when you have multiple developers or multiple projects on the test server.

= =
I guess the only 100%  way is to have a staging server where a deployment can be copied to and tested before deploying to the Live server - in this case it is tightly controlled and copy the Cache.dat is possible

Peter

Peter Cooper · Mar 18, 2018 go to post

Hi Evgeny

Fascinating conversation.....

I am aware of projections but don't use them in my systems.
I think there is some confusion when I use the term "Test" server  - in my usage this is used for End User Acceptance testing and there are usually more than one project/release undergoing End User Acceptance Testing at any one time - copying the Cache.dat would take over releases that are not ready to go.

= =

I guess it depends on the nature of the operation - I work for individual clients rather than having a monolithic product - and (as above) there will be several projects on the go at each time for each client - so what I do works for me.

If there is a possible problem with the compile (your projections) then, I think, the solution is a staging server - where individual releases are deployed to this and once proven that cache.dat is copied to the Live

My method works in my situation 
I guess there is no single "correct" solution that will work for all cases.

Peter

Peter Cooper · Mar 18, 2018 go to post

Hi Anne

Looks like we are on the same journey - have a look here on what I have so far

= =

I have no real answer to the logon issue - I think the "correct" solution is for Intersystems to provide a restful logon service to provide this functionality that can be accessed without being already logged on.

One possibility that I can think of that *may* work (have not tried it *yet*) ....
create a separate CSP web site that has no restrictions on logging on - with a single restful service "logon"  - in can be in the same namespace but with a different dispatch class.


The COS then performs a $System.Security.Login("username", "password") 
and returns success or fail - and sets up the session token

I can see that there may be issues with CORS - but I don't understand this topic yet :}

= =

As an aside....
I have been working with Caché for (too) may years and their security model/features is hidden away, difficult and subject to change.
I can understand this - they need to protect their commercial interests - eg it is technically possible for a minimal license (5 users) to support 1,000's of end user clients - clearly not commercially sensible.

Peter

Peter Cooper · Mar 18, 2018 go to post

Hey Eduard

This I understand and I use this at the moment in my ZEN apps
But...
Anne needs a JSON restful method of getting a logon and get a session token - only consuming a single license slot.
*NOT* a HTML web page to logon  that a CSP logon delivers - this is not what she is looking for.

As she said, you could pass CacheUsername and CachePassword as a URL parameter - but this don't work - if it fails Cahché delivers a HTML page - not what is required for a restful logon. 

The display/content should be in control of the client UI - not delivering a CSP HTML page

I'm sorry if I seem to be dismissive of your answer - but the web is moving on and we can no longer rely on the "old" way of doing things.

Peter 

Peter Cooper · Mar 19, 2018 go to post

Hi Evgeny

Source control library is not mine - It was a commercial product created by GlobalWare - saddly did not make it commercially and the company went defunct a few years ago.

But...
The main owner of the company now works for ISC in Boston wrc - Jorma Sunamo by name - maybe you should contact him direct (jorma.sunamo@intersystems.com) to discuss.
Peter

Peter Cooper · May 11, 2018 go to post

Hi Jeff 

and welcome aboard

If you want more (private) feedback please email me direct

Peter

Peter Cooper · May 11, 2018 go to post

Hi Darren

I remember an Intersystems Global Summit (or DEVCon and it was called back then) around 10 years ago where ISC did a presentation (Robert Nagle) on a  project where they got MIT to develop the same business app using open source and Caché.

It was big data where they were monitoring and capturing real-time data from solar energy installations (from small to large)

The project never fully completed but the indications were that the closed Caché outperformed open source on all metrics.

Interesting to see what's changed!
Just wish I had a copy of the presentation

Peter 

Peter Cooper · May 11, 2018 go to post

Hi Eduard

I had already seen your second comment elsewhere - but thanks for the additional detailed comment - most useful.

Just some quick thoughts......

  • All brokers effectively have Parameter UseSession = 1;
    But the default value in %CSP.REST is Parameter UseSession As BOOLEAN = 0;
    So the developer has to remember to override this every time (or sub-class)
     
  • Use same GroupById  
    I have seen this but have no understanding of what it's for - searching the docs only brings up a reference in the automatic deployment section only with no description
    If you have any more information on it's purpose please share 
     
  • Sharing the session between the CSP/ZEN app and Rest calls is a great feature
    But if it's a third party app then there is no CSP/ZEN app - the use case I have in mind is a 3rd party web developer is creating a complex shop system that needs to communicate with Caché
    I have no idea or interest in what technology they are using and it may be that their programming language does not easily support cookies so the CSPCHD (the session cookie) does not get passed.
    I am thinking that in this case the authentication needs to be passed with each Rest call - not an issue
    (or use OAUTH which I know little about)
     
  • On security
    Yes SQL security for sure - but there are traps here in the management of that security - I don't want a huge overhead in having to manage multiple roles each with it's own SQL security profile.
    And anyway it does not help if the Rest application is accessing the database via the object projection rather than SQL
    ​

As always, there are many options/scenarios my aim is not to give a definitive answer but to share my experience as a real world developer and how I implement things

Peter

Peter Cooper · May 11, 2018 go to post

Oh forgot this one......

Maintaining State between client calls...

I can think of a real world case where some sort of state between the client and the server is a good idea...

Consider a huge database where we need to query and produce the equivalent of the ZEN tablepane that displays rows of data with filtering and sorting functionality
I can see there are a number of widgets available for different frameworks where the whole result set is shipped to the client and filtering/sorting is done there.

However there are cases were....

  • We want the query to be run once on the server
  • We do not want  to ship the whole resultset  (too big there could be 100,000's rows) to the client and do pagination/sorting/filtering there
  • So the client component needs to get the data page by page from the server - and any filtering/sorting is done there
  • This is just how the ZEN tablepane works using ^CacheTemp(xxx to stash the data
  • Hence we are maintaining a state between the client and server

Unless you can suggest a better alternative that does not require state to be maintained

At the moment this is the only use case that I can think of where maintaining state is a useful thing 

Peter

Peter Cooper · May 11, 2018 go to post

Hi Eduard

Food for thought !!!!

It's just that it's always been the way I have done it in ZEN for around 11 years - started off with "small" result sets and just carried on doing it that way
It's great to be challenged - Thank you!!!

= =

BTW I like the look of the way that you include source code in your posts
Do you care to share how you create the posts with the grey box around them - do you create/edit with the online editor or some other tool?

Peter

Peter Cooper · May 11, 2018 go to post

Thanks again Eduard

Just one point it's Angular2 not AngularJS - I think AngularJS could be delivered by CSP but I doubt that it would be the best way to deliver Angular2 
Or so I believe - it's one of the things on my list to look at with Angular 2

Peter

Peter Cooper · May 12, 2018 go to post

No not intentional!

Everybody is good at some things and poor at others - and one thing I just *know* I am poor at is proof reading - I see what should be there rather than what actually *is* there

Thanks!

Peter

Peter Cooper · May 20, 2018 go to post

Hi Dimitry
But still inconsistent
eg
set x={"TheValue":"123"} write x.TheValue

Does not return a quoted string

Peter

Peter Cooper · May 20, 2018 go to post

Hi Demitri

It's me!!!!
I thought javascript was doing something different
if (oJSON.theValue) { write("hello") }

I *thought* that it was evaluating it as a boolean

It's not - the test is showing if it's defined

So.....
Learnt something today

Thanks
Peter

Peter Cooper · May 20, 2018 go to post

Hey Herman

Glad to see that you are still around!!!!
Yup it's me - I was not understanding - see my reply above

Peter

Peter Cooper · Jun 22, 2018 go to post

Hi Davidson

Please note $zf(-1 is now depreciated as of the most recent releases/patches of Caché

The replacement is $zf(100 - but stressing this will only work on the most recent patches

Peter

Peter Cooper · Jun 20, 2017 go to post

Hi Dan, Otto and Kyle

I disagree !!!

If you think about real world examples - eg invoice header and invoice lines

for me it makes no sense for invoice lines to exist without a header - I would want the db to enforce this as it does with parent child relationship

Agreed that the global buffers will have to load more data - but then again it's most likely that you want to see the lines whenever you see the header,

But I do agree with Otto re bitmap indices - but it has to be some huge database for this to impact on performance and if you are in this sort of size that it makes a diff then it's a different problem entirely and requires some deep thought re the design

Peter