Migrating to Giraffle for TigerGraph (Remote TigerGraph Part 1)

Introduction

In my last couple of articles, Full Stack TigerGraph Part 1 and Designing Feed Relationships with Graph Databases, I spoke about how to interact with TigerGraph from a Front End client as well as a use case of TigerGraph in a Social Media setting.

In this article, we will use Giraffle, a Gradle plugin which abstracts environment specific information including credentials for TigerGraph. Using Giraffle allows us to performs tasks from the CLI programmatically.

Agenda

• Giraffle Set Up
• Fetching TigerGraph Queries
• Fetching TigerGraph Schema

Giraffle Set Up

First, in order to use Giraffle, Gradle must be installed and set as a path variable on your computer. Check out this link to see how to set up Gradle. Moreover, if you would like to set Giraffle up with the official documentation, check out the documentation. Jon Herke also wrote a fantastic article on how to set up Giraffle. If you are interested in just setting up Giraffle, check out his article.

To check if the Gradle setup went correctly, Let’s first make the directory where Giraffle will be operating in. In the command line, migrate to the directory you have just made, and type in Gradle. If the output is similar to this, then Gradle is working!

Now that gradle is working, let’s move onto integrating the Giraffle Plugin.

In this same directory, let’s run the following command:

C:\Users\ramap\Documents\Medium\GiraffleDemo>gradle init

We receive the following prompt.

Let’s select basic and move along.

Here are what the rest of the prompts look like. Make sure you select Kotlin instead of Groovy.

Now we need to go to our folders and start editing some files. Here is my folder structure.

Let’s navigate to our build.gradle.kts file, and add the following plugins and dependencies in.

import com.optum.giraffle.tasks.*
import com.optum.giraffle.*
plugins {
id("com.optum.giraffle") version "1.3.4.1"
id("net.saliman.properties") version "1.5.1"
}
repositories {
jcenter()
}

As a confirmation, let’s run Gradle Tasks. In the same directory.

C:\Users\ramap\Documents\Medium\GiraffleDemo>gradle tasks --console=plain

Once it builds successfully, let’s run the gradle task gsqlNewProject. This task will setup our folder structure and files.

C:\Users\ramap\Documents\Medium\GiraffleDemo>gradle gsqlNewProject --console=plain

The Project Wizard will prompt you to enter information about your TigerGraph Instance.

We are almost done setting up Giraffle. We need to get a certificate from our instance. Simply run this command to fetch your certificate. You must replace hostname with your TigerGraph Instance host name.

openssl s_client -connect hostname.i.tgcloud.io:14240 < /dev/null 2> /dev/null | \
openssl x509 -text > cert.txt

Now that we have our cert.txt. Let’s go into our gradle.properties file. We must add a couple more properties. I believe the properties, gSecret, gClientVersion and gCertPath are optional, but let’s add them in.

Here are the updated properties:

gHost=
gHostUriType=
gAdminUserName=
gAdminPassword=
gUserName=
gPassword=
gGraphName=
gCertPath=
gClientVersion=
gSecret=

Make sure all these properties are in your gradle.properties file. Do not fill out these properties with their values in this file. We will fill that information out in the gradle-local.properties file. For example, here is how my gradle-local.properties file looks.

Make sure you do not check this file into version control.

Now that we have our gradle-local.properties, let’s head back to our build.gradle.kts. The Project Wizard may have overwritten our initial build.gradle.kts file. This is the file I am using.

The plugin “net.saliman.properties” is needed since we are using gradle-local.properties.

Just to confirm it is working, try running gradle tasks again. Once you run it you should see the following output.

Let’s try running the gsqlShell task. This task will open a gsql shell session.

C:\Users\ramap\Documents\OrgX\cloud\graph>gradle gsqlShell --console=plain

If the task is working, then you should get a message that says “Welcome to TigerGraph.”

Congratulations, Giraffle is set up and ready to go!

Fetching TigerGraph Queries

Now that the setup is finished. Let’s get those TigerGraph queries that are on the server into our own machine. To do this, let’s first enter into the GSQL Shell by running the gradle task:

gradle gsqlShell --console=plain

After you connect to the shell. Make sure you specify the Graph from which you want to pull your queries from with the following command.

use graph GRAPHNAME

Replace GRAPHNAME with your Graph

Now, simply run this command.

show query*

This command will print out the queries in the Command Line area. The idea is that we will take the output, put it in a text file and then parse the output and extract each individual query and make a file for it. The problem with this idea is that it may not print out all the queries depending on how many queries you have or how many lines each query is. For example, I have 40 queries, and they add up to 1000 lines. The shell command printed out 39 of the queries, nearly all of them. If there is an better solution, please let me know.

Now then, let’s create a file called queries.txt to put the printed text in. This is the script I used to parse each query and write it to a file:

If you are going to use this script, make sure you remove any partially printed queries from the gsql shell output.

The TXT_PATH should point to the file where the printed text was placed. The QUERY_PATH should point the folder where you want your queries to be. I recommend the db_scripts/query to be the folder where you put all your queries. Here is a reference.

This is my query from GraphStudio:

Here is how it looks after the process.

I added DROP QUERY and INSTALL QUERY so that we can try to directly install the queries from our remote machine in the next article. If you are going to use the script above to parse the queries, please be cautious. Make sure all the queries from your server came intact. If some of the queries did not show up, we must manually make the files for them.

Our queries are now migrated!

Fetching TigerGraph Schema

The idea for fetching the Schema is the same for getting the Queries. We will still need to be in the GSQL Shell and on the same Graph. However, let’s first create a file in db_scripts/schema called schema.gsql.

Now Run ls in the shell.

Under Vertex Types: all the Vertices will be printed. Copy the output of the vertices and place it into your schema.gsql file. For reference, here is how my schema.gsql file looks:

Select all occurrences of “- ” and replace it with “CREATE ”.

Copy the edges into schema.gsql as well. Repeat the same procedure for the edges.

Now, under Graphs: , copy paste the output for the Graph also in schema.gsql. Make sure to replace “-” with “CREATE” and replace all occurrences of “:e” and “:v” with an empty string.

Now our schema is also migrated.

Conclusion

The 2 main things, our queries and our schema, are now migrated to our local machine. In the next article, we shall explore how we can deploy multiple changes to our graph using GsqlTasks in Giraffle.

Please let me know if you liked or disliked some part of the article and how I can improve. Thank you for reading!

Ramko9999 - Overview

Here are some additional resources for Giraffle.

Optum/giraffle

Giraffle - Gradle Plugin for Tigergraph

Huge shout out to Joshua Meekhof, the author of Giraffle, and Jon Herke. Here is a video of them walking through using Giraffle on a Enterprise Scale.

Using Giraffle to Manage Deployments at an Enterprise Scale

Feel free to connect with me on LinkedIn

Rama Krishna Pitchala - Team Lead - Futurist Academy | LinkedIn