Trying to extract the narrative text from large CCDA files using EnsLib.EDI.XML.Document:GetValueAt method

The <MAXSTRING> error and <INVALID OREF> issues encountered while processing large CCDA files with the EnsLib.EDI.XML.Document:GetValueAt method can be attributed to string and object limitations. Here's a detailed response with potential solutions:

1. MAXSTRING Error when using GetValueAt Method
a. Reasons:

• If the data at the location specified in GetValueAt exceeds the maximum string length for %String, this error happens.
• The default maximum string length in InterSystems IRIS or Caché environments is 3.64 MB unless adjusted by enabling long strings.

b. Solution:

• Instead of extracting large strings directly, use stream objects. Create a stream from the raw data and operate on this stream to bypass the string size limitation:

  Set stream = ##class(%GlobalCharacterStream).%New()
  Do object.GetFieldStream("YourLocationPath", .stream)
  

• Ensure long strings are enabled in your IRIS configuration under System Administration to increase internal string size limits [1][2].

2. INVALID OREF Error during Conversion
a. Reasons:

• This error often arises when attempting to perform operations on an invalid object reference (OREF). Specifically, this could happen if the GetSubDocumentAt method isn't returning a valid object.

b. Solution:

• Always verify the object reference before attempting any further actions using the $isobject function:

  If '$isobject(subDocument) {
      Throw ##class(%Exception.InvalidOrefException).%New("Invalid object reference.")
  }
  

• Alternatively, use the ImportFromStream method of EnsLib.EDI.XML.Document if you'd prefer better memory-handling mechanisms:

  Set newDoc = ##class(EnsLib.EDI.XML.Document).ImportFromStream(rawStream, .status)
  If $$$ISERR(status) { Write "Error importing document!" }
  

3. General Recommendations for Extracting Narrative Text:

• If repetitive reads of large node content are required, consider writing a custom class/method to handle specific object-based operations. Streamline operations by processing larger XML structures into manageable chunks or storing specific values in temp files.
• For schema-based documents, verify that proper namespace handling or schema validation paths are followed [2][3].

For improvements in handling errors based on memory and OREF limitations, focus on preventive and refactored use of streams for large content, validated object references, and clear fallback strategies. [1][4][5][6]

Sources: