Hi all. We are going to find duplicates in a dataset using Apache Spark Machine Learning algorithms.

Note: I have done the following on Ubuntu 18.04, Python 3.6.5, Zeppelin 0.8.0, Spark 2.1.1

Introduction

In previous articles we have done the following:

• The way to launch Jupyter Notebook + Apache Spark + InterSystems IRIS
• Load a ML model into InterSystems IRIS
• K-Means clustering of the Iris Dataset
• The way to launch Apache Spark + Apache Zeppelin + InterSystems IRIS

In this series of articles, we explore Machine Learning and record linkage.

Hi all.

I want to insert my dataframe into InterSystems IRIS. So, I tried to do this:

df = spark.read.load("/home/imported-openssh-key/zeppelin-0.8.0-bin-all/bin/resultData3/DF.json", format="json")
df.write.format("com.intersystems.spark").\
option("url", "IRIS://localhost:51773/DEDUPL").\
option("user", "********").option("password", "********").\
option("dbtable", "try.test1").save()

And got this error:

Hi all. Today we are going to install Jupyter Notebook and connect it to Apache Spark and InterSystems IRIS.

Note: I have done the following on Ubuntu 18.04,  Python 3.6.5.

Introduction

Hi all. Today we are going to upload a ML model into IRIS Manager and test it.

Note: I have done the following on Ubuntu 18.04, Apache Zeppelin 0.8.0, Python 3.6.5.

Introduction

These days many available different tools for Data Mining enable you to develop predictive models and analyze the data you have with unprecedented ease. InterSystems IRIS Data Platform provide a stable foundation for your big data and fast data applications, providing interoperability with modern DataMining tools. 

Hi all. Today we are going to use k-means algorithm on the Iris Dataset.

Note: I have done the following on Ubuntu 18.04, Apache Zeppelin 0.8.0, python 3.6.5.

Introduction

Hi all. When I try to do this in Apache Zeppelin:

  This error appears:

My jdbc interpreter:

Hi all. Yesterday I tried to connect Apache Spark, Apache Zeppelin, and InterSystems IRIS. During the process, I experienced troubles connecting it all together and I did not find a useful guide. So, I decided to write my own.

Introduction