Hyper-Converged Infrastructure (HCI) solutions have been gaining traction for the last few years with the number of deployments now increasing rapidly. IT decision makers are considering HCI when scoping new deployments or hardware refreshes especially for applications already virtualised on VMware. Reasons for choosing HCI include; dealing with a single vendor, validated interoperability between all hardware and software components, high performance especially IO, simple scalability by addition of hosts, simplified deployment and simplified management.

Please excuse my ignorance. I am trying to identify what areas would be best to review in the System Dashboard (for Cache 2010.2) for performance issues with the database. It seems to be running slower than usual, but I am trying to find out the best way to go about identifying what the issue is.

The following are captures from the System Dashboard.

As always, thanks a lot for your help.

In this short article we talk about how to get Yape running in a docker container to avoid having to setup python on your machine.

It's been a while since the last article in this series, so let's recap quickly.

We talked about using matplotlib to create a basic graph. Afterwards we introduced dynamic graphs using bokeh. In the 3rd part we talked about generating heatmaps using monlbl data.

Hi, colleagues!

Consider you need to put some (less  than 1GB) data to an indexed array and do some data manipulations and calculations with it.

Should I prefer global or local for it?

Is there any general rule when local is faster than global and vice-versa?

Globals, these magic swords for storing data, have been around for a while, but not many people can use them efficiently or know about this super-weapon altogether.

If you use globals for tasks where they truly shine, the results may be amazing, either in terms of increased performance or dramatic simplification of the overall solution (1, 2).

What is APM?

I am talking about Application Performance Management at global summit, and several people have asked what that means so it is time for a bit of an explanation.

APM or Application Performance Management (sometimes referred to as Application Performance Monitoring) has a very good (if complicated) explanation on Wikipedia but to me it just means looking at performance from the users’ point of view and the level of service provided to them.

Beginning - see Part 1.

3. Variants of structures when using globals

A structure, such as an ordered tree, has various special cases. Let's take a look at those that have practical value for working with globals.

3.1 Special case 1. One node without branches

Quotes (1NF/2NF/3NF)^ru:

Every row-and-column intersection contains exactly one value from the applicable domain (and nothing else). The same value can be atomic or non-atomic depending on the purpose of this value. For example, “4286” can be

• atomic, if its denotes “a credit card’s PIN code” (if it’s broken down or reshuffled, it is of no use any longer)
• non-atomic, if it’s just a “sequence of numbers” (the value still makes sense if broken down into several parts or reshuffled)

This article explores the standard methods of increasing the performance of SQL queries involving the following types of fields: string, date, simple list (in the $LB format), "list of <...>" and "array of <...>".

Have you ever thought about leveraging IIS (Internet Information Services for Windows) to improve performance and security for your Caché web applications?  
Are you worried about the complexity of properly setting up IIS?

See the webinar Configuring a Web Server presented by @Kyle Baxter,  InterSystems Senior Support Specialist. Learn how to install IIS, set up it up to work with the CSP Gateway, and configure the CSP Gateway to talk to Caché.

If you have not subscribed to our Developer Community YouTube Channel yet, let's get started right now.  

Enjoy!

Code coverage and performance optimization of code has come up a bunch of times already, so most of you should already be aware of the SYS.MONLBL utility. Often a visual approach to looking at code is much more intuitive than pure numbers, which is pretty much the whole point of this article series. This time we will take a slight excursion away from python and its tools and are going to explore generating heatmaps from ^%SYS.MONLBL reports.

 It's almost time to get your customers upgraded to new versions - are you worried about showing off your SQL Performance after upgrades?  If you want to upgrade without worrying, then I have just the program for you!!!  Check out this video from Global Summit 2016 featuring yours truly explaining how to upgrade a system without worrying about pesky SQL queries showing on your waistline!  

https://www.youtube.com/watch?v=GfFPYfIoR_g

Unfortunately the video started after the Frozen Musical Sing-a-long, but it's 30 minutes of the most fun you'll have while learning tools and tips for Caché SQL!

I have been asked to assist in the planning of the a new server for our database, which we will be changing operating systems from OpenVMS to Linux (RedHat distribution). However, its difficult to find material regarding what would be recommended, which is likely due to the database being proprietary.

In looking at the information provided below and hoping to decrease processing time, would anyone be able to recommend type of configuration we should have for the new Linux server? Please feel free to ask any clarifying questions.

Any recommendations, to this VMware disk type debate, with Cache installations in particular?

Thick Provision Lazy Zeroed

Thick Provision Eager Zeroed

Thin Provision

While I can see the benefits that $ZSTORAGE could have if used properly, I have not seen it used in the environments I have worked in. I was wondering if there are any developers that promote its usage.

If used properly, I would imagine it could be highly effective in maximizing free memory since some processes will never go over X amount, while others may very well need much more.

Can someone direct me to where in the documentation we can find how consumption may be calculated for global storage?

EDIT: After receiving some responses, it seems I was unclear in my initial inquiry. I am looking to determine our rate of consumption of storage; however, I am having some difficulty in doing that.

While utilizing ^%GSIZE, which is used by the %GlobalEdit class, the results appeared odd. I have provided my results below, which illustrate the global structure on the left and the usage indicated by ^%GSIZE on the right.

I was benchmarking the populate utils when I noticed something strange. Consider this simple benchmarking method:

One of the great availability and scaling features of Caché is Enterprise Cache Protocol (ECP). With consideration during application development distributed processing using ECP allows a scale out architecture for Caché applications. Application processing can scale to very high rates from a single application server to the processing power of up to 255 application servers with no application changes.

We don't often use SQL within our org, which is mostly due to the performance issue we experience due to the quantity of data we are reviewing.

Aside from the standard performance measures for non-Caché databases, are there any recommended approaches when querying large tables?

The table would have roughly 50M records, but there are not a finite amount of sub-nodes.

Index

This is a list of all the posts in the Data Platforms’ capacity planning and performance series in order. Also a general list of my other posts. I will update as new posts in the series are added.

You will notice that I wrote some posts before IRIS was released and refer to Caché. I will revisit the posts over time, but in the meantime, Generally, the advice for configuration is the same for Caché and IRIS. Some command names may have changed; the most obvious example is that anywhere you see the ^pButtons command, you can replace it with ^SystemPerformance.

While some posts are updated to preserve links, others will be marked as strikethrough to indicate that the post is legacy. Generally, I will say, "See: some other post" if it is appropriate.

Capacity Planning and Performance Series

Generally, posts build on previous ones, but you can also just dive into subjects that look interesting.

• Part 1 - Getting started on the Journey, collecting metrics.
• Part 2 - Looking at the metrics we collected.
• Part 3 - Focus on CPU.
• Part 4 - Looking at memory.
• Part 5 - Monitoring with SNMP.
• Part 6 - Caché storage IO profile.
• Part 7 - ECP for performance, scalability and availability.
• Part 8 - Hyper-Converged Infrastructure Capacity and Performance Planning
• Part 9 - Caché VMware Best Practice Guide
• Part 10 - VM Backups and IRIS freeze/thaw scripts
• Part 11 - Virtualizing large databases - VMware cpu capacity planning

A short post for now to answer a question that came up. In post two of this series I included graphs of performance data extracted from pButtons. I was asked off-line if there is a quicker way than cut/paste to extract metrics for mgstat etc from a pButtons .html file for easy charting in Excel.

See: - Part 2 - Looking at the metrics we collected

This is a follow-up to Murray's article for scripting pButtons data extraction on Unix/MacOS:     Extracting pButtons data to CSV for UNIX/MacOS

###Extracting pButtons data to a CSV file on Windows

PowerShell scripting was used in this article because it is available by default on Windows platforms.

The script, Extract_pButtons.ps1, will search the given pButtons file until it finds the marker for the beginning of the section. The section will be printed line-by-line, from the header to the line before the section end marker.

It has been noticed that some customers running JAVA programs (for example, FOP) on AIX would see the server eventually running low then out of memory. Customer would notice the system pages heavily and user experience becomes bad. And the server would crash when out of memory.

When the problem happens, we can see in ipcs a lot of shared memory segment marked for deletion (Capital D at the beginning of MODE section). This means they will not disappear until the last process attached to the segment detaches it.

I saw someone recently refer to ECP as magic. It certainly seems so, and there is a lot of very clever engineering to make it work. But the following sequence of diagrams is a simple view of how data is retrieved and used across a distributed architecture.

For more more on ECP including capacity planning follow this link: Data Platforms and Performance - Part 7 ECP for performance, scalability and availability

To start

Date range queries going too slow for you?  SQL Performance got you down?  I have ANOTHER weird trick that SQL Developers don't want you to know!*

Often times support and sales engineers are asked about recent benchmark results on various platforms and large scale configurations.  These will be made available here in the Developer Community in the "Documentation" section, and as an example here's a link to a recent Intel E7 v2 series processor benchmark.

https://community.intersystems.com/documentation/data-scalability-intersystems-caché-and-intel-processors-0

There are several reports available and more will be made available on an on-going basis.