Embedded Python and IRIS Streams
Hi All,
We're doing our first babysteps with embedded Python and IRIS with an interoperability solution. We want to convert a CSV file to an Excel xlt file.
As service we've got a EnsLib.File.PassthroughService which picks up a csv file
As operation we've got a EnsLib.File.PassthroughOperation which writes the Excel file.
In the middle:
We've created an Business Process with an %Stream.GlobalCharacter in the Request and a %Stream.GlobalCharacter in the Response.
But how do we get the %Stream.GlobalCharacter in the Python ClassMethod? Ideally I don't would like to write the stream first to a temp file, give the temp file to Python and let Python write the XLT file and read that XLT file into a %Stream.GlobalCharacter.
Any ideas? See the code snipped below. This one works but will first create a string from the Stream but will fail when the csv is bigger then the Intersystems string limit
Class ZORG.BP.CsvToExcel Extends Ens.BusinessProcess
{
Method OnRequest(pRequest As ZORG.BP.CsvToExcel.CsvToExcelReq, Output pResponse As ZORG.BP.CsvToExcel.CsvToExcelResp) As%Status
{
Set tsc = $$$OK; Create the response classSet pResponse = ##class(ZORG.BP.CsvToExcel.CsvToExcelResp).%New()
Set pResponse.outputStream = ##class(%Stream.GlobalCharacter).%New()
$$$TRACE("Size of Stream: "_pRequest.inputStream.SizeGet())
Set XlsString=..convertCSVtoXLS(pRequest.inputStream.Read($$$MaxStringLength))
Do pResponse.outputStream.Write(XlsString)
return tsc
}
ClassMethod convertCSVtoXLS(csvFile As%String(MAXLEN="")) As%String [ Language = python ]
{
"""
This function takes a CSV string as input and converts it into an XLS string.
Parameters:
csv_string (str): The CSV string to be converted
Returns:
str: Th
"""
import pandas
import xlwt
import sys
import csv
import tempfile
from io import StringIO
# Read the CSVdata into a Pandas DataFrame
df = pandas.read_csv(StringIO(csvFile), delimiter=";", decimal=",", dtype={'patno':str} )
# Create an instance of Workbook class from xlwt library
workbook = xlwt.Workbook()
# Add a sheet to the workbook
sheet = workbook.add_sheet('Sheet1')
# Write the DataFrame values to the sheet
for i, column in enumerate(df.columns):
sheet.write(0, i, column) # Write column headers
forj, value in enumerate(df[column]):
if ( str(value) != 'nan' ): #catch empty cells
sheet.write(j + 1, i, value) # Write cell values
# Write workbook to temporary file
file = tempfile.TemporaryFile()
# Save Workbook
workbook.save(file)
#Go to the beginning of the file
file.seek(0)
#Read from temporary File and return
data = file.read()
return(data)
}
}
Comments
That how i read Stream and Write stream with Embedded Python :
Read Stream :
defstream_to_string(stream)-> str:
string = ""
stream.Rewind()
whilenot stream.AtEnd:
string += stream.Read(1024)
return stringWrite Stream :
defstring_to_stream(string:str):
stream = iris.cls('%Stream.GlobalCharacter')._New()
n = 1024
chunks = [string[i:i+n] for i in range(0, len(string), n)]
for chunk in chunks:
stream.Write(chunk)
return streamHello Guillaume,
Thank you for the example code.
Will give it a try later on this week and will post the new code.
Menno
@Guillaume Rongier, I've had some struggles with converting but that was nog due to your functions.
See a snippet of working code below:
ClassMethod Decrypt(Stream As%Stream.GlobalCharacter, key As%String) As%Stream.GlobalCharacter [ Language = python ]
{
import iris
import os
from cryptography.hazmat.primitives import hashes, padding
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.backends import default_backend
from base64 import b64decode
def stream_to_string(stream)-> str:
string = ""
stream.Rewind()
while not stream.AtEnd:
string += stream.Read(1024)
return string
def string_to_stream(string:str):
stream = iris.cls('%Stream.GlobalCharacter')._New()
n = 1024
chunks = [string[i:i+n] for i in range(0, len(string), n)]
for chunk in chunks:
stream.Write(chunk)
return stream
# Convert the Base64 encoded key to bytes
key_bytes = b64decode(key)
cipher_bytes = bytes(stream_to_string(Stream),'iso-8859-1')
# Extract the IV from the first 16 bytes of the cipher
iv = cipher_bytes[:16]
# Create the AES cipher object
backend = default_backend()
cipher = Cipher(algorithms.AES(key_bytes), modes.CBC(iv), backend=backend)
decryptor = cipher.decryptor()
# Decrypt the data, excluding the IV
decrypted_bytes = decryptor.update(cipher_bytes[16:]) + decryptor.finalize()
# Remove padding
unpadder = padding.PKCS7(algorithms.AES.block_size).unpadder()
decrypted_data = unpadder.update(decrypted_bytes) + unpadder.finalize()
return string_to_stream(decrypted_data)
}