magnolia

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 8 Open GitHub ↗

PubSub utilities and dynamic plugin testing for neo4j

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Magnolia
===========

This repo is a collection of neo4j plugins and experiments of mine.  Right now there are two somewhat different
things in here.

(1) GraalVM support, and polyglot functions/procedures for cypher.  This lets you write cypher procedures in
javascript and python, and also do hot-reloading, so you can change the code of a procedure without server restart
or a new plugin.

(2) Google PubSub support.  Neo4j can either have transaction hooks in place and publish new nodes/edges to
a PubSub topic, or it can selectively publish only the results of a cypher query, or only certain nodes/edges.

# GraalVM Support

## Setup

- Download the latest community graalvm, and unzip it in this directory.
- (Optional) Run `./graalvm-ce-1.0.0-rc5/Contents/Home/bin/gu install python` to install python support.
- Configure your IDE to use GraalVM in that path as the JDK for building this project.
- Build the code `mvn clean package -DskipTests`
- Build the neo4j-graal docker VM locally:

cd neo4j-graal
    docker build -t neo4j-graal:3.3.5-enterprise -f Dockerfile .

- Run the "create-empty.sh" script to run the built docker container locally with the magnolia plugin.
- Connect cypher-shell to localhost (username and password are in create-empty.sh), and then try to run
`CALL magnolia.list()` or other procedure, see examples below.

## Configure Magnolia for Yourself

You can examine the `create-empty-db.sh` script for examples of how to run with a configuration.

Write a `magnolia.yaml` file that looks like this:

basePath: "/plugins/magnolia"
    procedures:
      - name: my-proc
        file: my-proc.js
        language: js
    functions:
      - name: my-function
        file: my-function.js
        language: js

The file `my-fuction.js` will be expected to be found under `basePath`.

Set the magnolia configuration in neo4j by using the following in neo4j.conf:

`magnolia.configuration=/path/to/magnolia.yaml`

The purpose of setting configuration to be in a separate yaml file, and not entirely in neo4j.conf
as is normal with other plugins, is to allow dynamic reloading.  In this way, you can change magnolia.yaml, and 
even add functions, without restarting neo4j.

**Recommended**: create a directory under the plugins directory where you install neo4j, and put all of your
extension scripts, and magnolia.yaml file there.  Ensure everything magnolia needs is under the basePath
specified by

## Examples of Using Magnolia

### List Configured Functions

The `list()` procedure returns the available registered dynamic functions & procedures.

### Call a Registered Function

The `magnolia.fn` cypher function takes two arguments; the name of the dynamic function you want to 
execute, and an argument to that function (which may be a map or other neo4j structure)

neo4j>     CREATE (p:Person { name: "Emil" })
               RETURN magnolia.fn('echo', "Hello, " + p.name) as greeting;
    +---------------+
    | greeting      |
    +---------------+
    | "Hello, Emil" |
    +---------------+
    
    1 row available after 77 ms, consumed after another 0 ms
    Added 1 nodes, Set 1 properties, Added 1 labels

### Call Dynamic Function

The `magnolia.dynamic` function allows you to put javascript directly into cypher.  In the function
example below, "Interop.import" is how GraalVM takes arguments from the guest language, and the final
argument "5" is what's bound to "arguments" inside of the javascript code.  The first argument ("js") simply
indicates that the code is intended as javascript.

neo4j> WITH 'const input = Interop.import("arguments"); input * input' AS javascriptSquareInput
           RETURN magnolia.dynamic('js', javascriptSquareInput, 10);
    +---------------------------------------------------+
    | magnolia.dynamic('js', javascriptSquareInput, 10) |
    +---------------------------------------------------+
    | 100.0                                             |
    +---------------------------------------------------+

# Google PubSub Support

Magnolia also contains a Neo4j server plugin for streaming node and edge messages to pub/sub
connectors, and other utilities useful for testing.

## Why?

Connections through messaging systems are important for implementing layered architectures; for example,
you can connect existing ElasticSearch systems to PubSub, so by publishing to PubSub, you can easily set
up database replication to ElasticSearch and add search to an existing graph database.

## Setup

1. Build it:

mvn clean package -DskipTests

2. Copy target/magnolia*.jar to the plugins/ directory of your Neo4j server.

3.  Configure your Neo4j server:

```
magnolia.pubsub.project=my-google-project-id
magnolia.pubsub.topic=some-google-pubsub-topic-id
```

4. Start your Neo4j Server

5. Run any sample cypher to create some data:

```
CREATE (p1:Person { name: "David" }),
(p2:Person { name: "Mark" }),
(p3:Person { name: "Susan" })

MERGE (p1)-[:KNOWS]->(p2)
MERGE (p2)-[:KNOWS]->(p3)
MERGE (p3)-[:KNOWS]->(p2);
```

6. If you have an existing pull subscription on the topic, you should see messages published.  Messages look like this:

Nodes:
```
{ 
  "entityType":"node",
  "id":0,
  "event":"create",
  "properties":{"name":"David"},
  "labels":["Person"]
}
```

Relationships:
```
{
  "entityType":"relationship",
  "start":2,
  "end":1,
  "id":2,
  "event":"create",
  "type":"KNOWS",
  "properties":{}
}
```

magnolia

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README