Sharding is a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts called data shards. The word shard means a small part of a whole.
What are the best practices for choosing a shard key in InterSystems IRIS to ensure long-term scalability without needing major refactoring later?
In this article I'll show you how to set up in your laptop, very quickly, a cluster of IRIS nodes in sharding. It's not the goal of this article neither to talk about sharding in detail nor define a deployment of a production ready architecture, but to show how to set up quickly, in your own machine, a cluster of IRIS instances configured as shard nodes, with which you'll able to play and test this functionality. If you're insterested in knowing more about sharding in IRIS, take a look at the documentation clicking here.
Hi Community!
Please welcome a new video on InterSystems Developers YouTube Channel:
I found a package on OEX for a Sharding Demo
If Sharding is NOT included in the Community License
I can not use the Community Distribution but require a different one.
And have to add ZPM manually.
In the good old days (tm) determining the size of the data, streams, and indices for a class/table was easy - you just ran %GSIZE and check D, S, and I globals respectively.
However, nowadays sharding, optimized global names, and indices in separate globals produce %GSIZE output looking like this:
In this article, we will run an InterSystems IRIS cluster using docker and Merge CPF files - a new feature allowing you to configure servers with ease.
On UNIX® and Linux, you can modify the default iris.cpf using a declarative CPF merge file. A merge file is a partial CPF that sets the desired values for any number of parameters upon instance startup. The CPF merge operation works only once for each instance.
We are seeking experience or best practice for Sharding on IRIS and doing some testing on 2021. Appreciate if anyone can share your stories here. Thx!
Can InterSystems IRIS support horizontal expansion? How many nodes can it support？
Hi Community!
Enjoy watching the new video on InterSystems Developers YouTube:
Hi Community,
Please welcome the new video from Global Summit 2019 on InterSystems Developers YouTube:
Prior to IRIS, using ECP to share databases under heavy I/O load has known latency issues (in our environment), which pretty much restricted using shared database with relatively static or slow-changing data. In IRIS, will sharding mitigate the latency issue and allow any table to be shared?
David
Allow a question from a newbie
1. Lets assume we have a global (matrix) [X,Y,Z] that is distributed across sharded nodes
2. Matrix size doesnt matter, but lets assume it holds 500 GB for the moment
3. I want to return all rows where f(x,y,z) is true. f() is an arbitrary function, i.e. f = x + 20(y*y) > z
Questions
Q1. Can such a computation (in ObjectScript) be moved and distributed from the client to the different sharded nodes for parallel processing
Q2. Where is that documented (incl. perhaps an example and performance measurements)
Thanks a lot
Frank
InterSystems and Intel recently conducted a series of benchmarks combining InterSystems IRIS with 2nd Generation Intel® Xeon® Scalable Processors, also known as “Cascade Lake”, and Intel® Optane™ DC Persistent Memory (DCPMM). The goals of these benchmarks are to demonstrate the performance and scalability capabilities of InterSystems IRIS with Intel’s latest server technologies in various workload settings and server configurations. Along with various benchmark results, three different use-cases of Intel DCPMM with InterSystems IRIS are provided in this report.
InterSystems Online Learning has updated courses to include the latest information about InterSystems IRIS. See descriptions below, and try a course today!
Troubleshooting Productions
Get an introduction to troubleshooting productions, with a focus on locating and understanding some of the key Management Portal pages when troubleshooting.
Hi Everyone!
New video, recorded by @Benjamin De Boe, is already on InterSystems Developers YouTube:
Hi Community!
Please welcome a new video on InterSystems Developers YouTube Channel:
The preview release of InterSystems IRIS 2019.2 is now available - give it a try!
Container images are available via the WRC's preview download site.
The build number for these releases is 2019.2.0.100.0.
InterSystems IRIS Data Platform 2019.2 is the first CD (continuous delivery) release of InterSystems IRIS. It has many new capabilities including:
Hi Community!
New session recording from Global Summit 2018 is available on InterSystems Developers YouTube Channel:
Hi Community!
New video "Choosing a Shard Key" is available now on InterSystems Developers YouTube:
Hi Community!
Please welcome a new video on InterSystems Developers YouTube:
Hi Community!
New video "Deploying Shards Using InterSystems Cloud Manager" is available now on InterSystems Developers YouTube:
In a single server environment, it is rather easy to create a transaction consistent backup.
Simplified:
You find (or force a specific point in time with no open transactions.
Freeze your server, take a snapshot and you are done.
Hi Everybody!
We are pleased to invite you to the upcoming webinar "Sharding as the basis of Scaling in InterSystems IRIS Data Platform" on 24th of July at 10:00 (Moscow time)!
With the release of InterSystems IRIS, we're also making available a nifty bit of software that allows you to get the best out of your InterSystems IRIS cluster when working with Apache Spark for data processing, machine learning and other data-heavy fun. Let's take a closer look at how we're making your life as a Data Scientist easier, as you're probably already facing tough big data challenges already, just from the influx of job offers in your inbox!
Last week, we announced the InterSystems IRIS Data Platform, our new and comprehensive platform for all your data endeavours, whether transactional, analytics or both. We've included many of the features our customers know and loved from Caché and Ensemble, but in this article we'll shed a little more light on one of the new capabilities of the platform: SQL Sharding, a powerful new feature in our scalability story.