#Journaling

I am attempting to failover my TEST environment to the Backup, and I keep getting an error stating that 

06/01/25-18:47:54:516 (11864) 1 [Utility.Event] Primary startup failed, failed to read header of /archive/journal/MIRROR-IRISTEST-20250513.007 (file #63653)

however, when I go to /archive/journal/, I am not finding any record of MIRROR-IRISTEST-20250513.007 (file #63653)

to get out of this error message I have to restart what was the Primary and restart the Backup to get it back into a state that we can use it. 

InterSystems has been at the forefront of database technology since its inception, pioneering innovations that consistently outperform competitors like Oracle, IBM, and Microsoft. By focusing on an efficient kernel design and embracing a no-compromise approach to data performance, InterSystems has carved out a niche in mission-critical applications, ensuring reliability, speed, and scalability.

A History of Technical Excellence

InterSystems has corrected a defect that causes invalid database and journal records to be introduced when using a specific $LIST syntax. The likelihood of encountering this defect is very low but the operational impacts can be significant.

Products Affected

What is Journaling?

Journaling is a critical IRIS feature and a part of what makes IRIS a reliable database. While journaling is fundamental to IRIS, there are nuances, so I wrote this article to summarize (more briefly than our documentation which has all the details) what you need to know. I realize the irony of saying the 27 minute read is brief.

InterSystems FAQ rubric

If the system does not stop for 24 hours, old journal files will be deleted at 0:30 according to the "Journal file deletion settings".

A possible cause of journal files remaining that are older than the "Journal file deletion settings" is that there are transactions that remain open.

In that case, you will be able to delete the journal file by searching for processes executing transactions and finalizing the transactions.

The sample below checks for the existence of open transactions, and if they exist, outputs the target file name and journal record information.

Hi community!

In this article, I'm excited to introduce my latest application, Journal File Indexer. The development of this application is rooted in the portal idea  DPI-I-270.  

In short, this application lets you load and index a log file in a database.  

Why?

I'm working for an organisation that is running a very old version of InterSystems Cache (5.016) which runs on AIX . The last two times we have re-booted Cache, we have encountered rollbacks. I've been asked two questions. During the rollback it was "How long is it going to take?" and after the system returned, it was "So what caused it?". My answer to both was "I don't know".

Hi!

Currently, I'm working on a script using %SYS.Journal API.

My journal files are compressed, following the documentation the compression is zstd.

I tried to unzip a file with zstd tool, ex: 

apt-get update
apt-get install zstd

zstd -d 20230806.001z -o 20230806.001

20230806.001z        : 92.03%   (909312 => 836859 bytes, 20230806.001) 

Unfortunately, the unzipped file is invalid:

Write##class(%SYS.Journal.File).IsValidJournal("/usr/irissys/mgr/journal/20230806.001")

I have several 1GB journals from a LIVE server that I would like to inspect (eg: check which globals have been updated over the time).

Is there a simple way to view those journals using another IRIS instance ? (eg: local installation).

I have been tempted to put those files directly into the journal folder of my local installation and restart the system, however I am concerned that the transactions they contains will be restored and will corrupt the local database.

I have checked documentation but couldn't find anything.

Recently I needed to restore a version of a production class, which was overwritten by compilation and running UpdateProduction. As the correct version was unavailable in the source control, I used journals to restore the data. Journals store a plethora of information about what's happening in the system and are quite a powerful tool. This article explains how to work with journals to extract the data you require.

Coming back to the topic of how to minimize journals.

In the Windows Ressource Manager I can observe multiple parallel processes coming from cache.exe with read operations to journaling files.

All except one of these processes have the same reads(Byte/s). The processes point to different journal files and constantly read between 200 and 3000 Bytes/s.

The corresponding process via PID in the management portal of Caché shows the process %SYS.Monitor.Control.1. In 3 days of uptime on the server it has run 181.632.583 commands and modified 32.140.642 globals. 

Hi All,

Our mirrored HealthShare environment has failed over a few times recently due to underlying infrastructure issues (that are being worked on and resolved).

In the HealthShare logs we are seeing:

10/06/22-00:54:35:925 (4736) 1 Journal Daemon has been inactive with I/O pending for 10 seconds:
gjrnoff=524741316,iocomplete=523852600,filecnt=1011,fail=0
10/06/22-00:54:55:086 (4736) 3 CP: Pausing users because Journal Daemon has not shown
    signs of activity for 30 seconds. Users will resume if Journal Daemon is active again

My question is:

Hello everyone,

I want to restore a database from an external backup with the journal files.

The manual says I should (in short):

1. Stop journaling with ^JRNSTOP
2. Restore the database file IRIS.DAT
3. Run the journal restore utility with ^JRNRESTO
4. Restart journaling ^JRNSTART

Since I want to restore one database only step 1 would disable journaling for the whole instance, not only for the database to be restored. As I understand, it would be impossible to restore a different database if there occurs a problem while journaling is disabled during the recovery process?

I need to turn off Journalling for a particular database programmatically.

How can I make it happen?

Is View Journal of Cache Management Portal the only way to search in Cache journals? I need a way to define several search conditions and Portal often breaks on http timeout.

Working in support, I usually get asked how many days I should keep journals. Should it be two days or after two backups? More? Less? Why two?

The correct answer (for most of the environments) is that you should keep the journals since the last validated Backup. I.e., until you don't check if a Backup is valid (restoring the file and checking with the Integrity utility), you can't be sure there is a good copy of your data and can't purge the journals safely.

I am new to Cache, we are trying to move a 4 TB database over the internet, but this will be take too long to copy the single backup file. In Oracle and MS-SQL Server there is an option of doing log shipping. Is there such an option in Cache?

I want to copy the initial backup file one weekend, then keep sending the Journals (logs) the new location.

Thank you,

Eudoro

Hey everyone.

I have noticed that my backup mirror is warning that the MirrorDatabaseLatencyTime is having a bad time (time in ms is 3000, and warnvalue is 3000). While I look into what may be causing this latency between the two servers, I was considering if reducing the size of the journal files would improve this value in any way.

My assumption is that reducing the file size would mean that the frequency of the journal files being created would be increased, but the reduced size would mean that the transfer and application of each file would be reduced.

Has anyone had experience in this area?

I see command shortcuts for getting journal details, like Status^JOURNAL for displaying the journal status.

And, I'm using the shortcuts in my shell scripts.

I'm not seeing/finding command shortcuts for getting database details.

If anyone have those details, please share with me.

Thanks,

Bharath Nunepalli.

Hi,

We are using CACHE 2017.2.1, I would like to retrieve data from Journal for killed global. Let say we have global name ^ EMP(123) with data and also have some child nodes and it has been killed by using cache kill command for some reason and we don't know who has executed this and when. My questions are below.

1) Can we get back the data of killed global from journal files,Is it possible or not ?

In this article I'd like to share with you a phenomena that is best you avoid - something you should be aware of when designing your data model (or building your Business Processes) in Caché or in Ensemble (or older HealthShare Health Connect Ensemble-based versions).

Hi, 

I'm having a problem when I'm attempting to freeze the instance.

I have a pre-script and post-script to freeze and unfreeze the instance, but when the script execute: 

 csession INSTANCE_NAME -U '%SYS' "##Class(Backup.General).ExternalFreeze(\"$SNAPLOG\",,,,,,1800)"

 It's give me this error and fails the freeze: 

Backup.General.ExternalFreeze: Failed, Unable to switch local journal file, Error: -99,ERROR #1142: Error switching journal file: 0
vi

At cconsole.log file show this:

Sorry about my newbie question.

How do you decide the size to give to journal1 and journal2?

PS: I am still in version 2016

Thank you

Prmoetheus is a really good to combine measures, and export them in graphical dashboards

https://prometheus.io/

http://grafana.org/

I would like to write some a caché exporter for Prometheus, that will allow to expose some metrics of the database, or even some metrics of my own

Did anyone already tried ?

Thank for the answer

I had my journal files all on one disk which died :( 

The database files are all fine, but I having issues.  Can I bypass the need for the journal files?

My fantasy didn't go beyond periodically running %SYS.Journal.System:Progress() class query, while I'd prefer to use an event handler of some kind. Any ideas?

I am still working on a generic task where I need to apply journal file records to another database. Initially I didn't want to use Journal.Restore class methods as I need to perform some data transformation, and it seemed that the clearest way to achieve it was to read journal file record by record using %SYS.Journal.Record API. 

This approach worked (with some help from @Dmitry Maslennikov and @Eduard Lebedyuk), while it turned that the processing speed of %SYS.Journal.Record:List query was very slow, about 1MB of journal data per second on a mid-range server.

I'm a DBA and support Caché databases on AIX. I coded shell scripts for monitoring journaling status, databases size, license end date.

We recently got a new instance of Caché on Windows. I'm just curious to know whether anyone coded database monitoring scripts on Windows using PowerShell or any other scripting language.

If yes, please share the details.

Thanks & Regards,

Bharath Nunepalli.

Just wondering if anyone knows if there is a way to get daily emails or alerts about changes to a cache database.  I know that all of that information is contained within the journal files, just wondering if there is a way to bundle it up each day and send it off for auditing, etc.

Any solution where a plain-text/readable format of changes to the cache database could be sent or stored for review would solve the issue.

Thanks!