How to avoid <STORE> error on this special example
Hello Community,
I am working on creating a FHIR bundle with Practitioner resources inside of a BPL business process. Data source is a cache database table containing business partner data objects.
Part of the implementation is a loop over all new or updated data objects. For each element a SAP call is made to retrieve more information and add it to the target resource. As there was no last run on first start, approx. 130000 objects are initially to be exported. After the first run, there would not be more than 100 to be updated on each run.
The process runs fine for some thousand objects, but then suddenly error messages occurr.
ERROR <Ens>ErrException: <STORE>%0Ao+4^%sqlcq.UKERFHIR.xEHqZZYvgEEsieFzA5g4Xbi2SJ2O.1 -- logged as '-' number - @''What I do in detail is
1. read all objects from the chache database into a %Library.ListOfObjects
2. loop over all objects
a. dtl transform object into a (new) fhir resource
b. dtl transform object into a request object
c. call the sap system and receive response object
d. dtl transform response object into (existing) fhir resource
e. put fhir resouce into another %Library.ListOfObjects
( f. target list unswizzle target fhir resource )
( g. source list unswizzle source object )
3. Create fhir bundle from target list.
4. Send bundle to fhir server
In the documentation I found a section which described something very similar to my kind of problem:
https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cl…
Inspired by this information, I added line g.) to the process, because I thougt this might be the right place to free space, but that did not help. Then I added f.) and got another error message
ERROR #5002: ObjectScript error: <METHOD DOES NOT EXIST>%UnSwizzleAt+4^%Library.ListOfObjects.1 *%GetSwizzleObject,HS.FHIR.DTL.vR4.Model.Resource.PractitionerI'm not sure if I fully understand the concept of freeing memory in this context. I also thought about splitting up bundle creation and external data retrieval in two different processes, as it might not be the best approach to process all objects at once.
What would be the best way to get rid of this error message, or at what point could I free memory to eliminate the problem?
Thanks for your time, I appreciate any help,
regards,
Martin
Comments
%GetSwizzleObject - is basically belongs from the %Persistent Class. The HS.FHIR.DTL.vR4.Model.Resource.Practitioner is extending from the %Library.RegisteredObject class. For the store error you can try increasing the process memory allocation programmatically to maximum by set $zstorage=-1 for the specific process.
Try %Reload() on the object holding the %Library.ListOfObjects after step 1. If it doesn't work, just kill the reference and %Open() it again. After each loop, do the same to the object holding the other %Library.ListOfobjects. I hope this helps.
Thank you very much for your feedback. After some back and forth and failed attempts, I have now decided on a different approach. The reasons for this are on the one hand that an export of all 133500 records would take something like 12h and in case of an error all preliminary work is gone.
Furthermore, even with 1000 records the transfer to a FHIR bundle again ran into a STORE error. All in all it is more reasonable to split the export into smaller packages. I got a handle on the problem by now initiating export at 500 objects and persistently storing the IDs of the handled records so they can be ignored on the next run.
The process is called on a timed basis every 5 minutes and works in chunks until all resources are exported. This runs since 1h now, and looks good so far.
Regards, Martin