Maybe it can help,
In the SYS.Database Class you can find the IsSystemDB(Directory) method that returns whether it is a system database.
It works on Caché and IRIS.
Documentation IsSystemDB - IRIS
Documentation IsSystemDB - Caché
Examples:
%SYS>set databaseDir = ##class(Config.Databases).Open("IRISLIB").Directory %SYS>write ##class(SYS.Database).IsSystemDB(databaseDir) 1
%SYS>set databaseDir = ##class(Config.Databases).Open("CACHELIB").Directory %SYS>write ##class(SYS.Database).IsSystemDB(databaseDir) 1