KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agent #developer-tools #codex #typescript #cli #deepseek-harness #claude-code #ai-agents #mcp #rust

laravel-crate.io

★ 36 Open GitHub ↗

Crate.io driver for Laravel

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Crate.io driver for Laravel

This is an Eloquent and Query builder support for Crate.io. Extends
the original Laravel API with Crate PDO driver.

### Crate and Crate PDO

Crate is a distributed SQL Database based on Elasticsearch, Lucene
and other goodies. See their official page on [Crate.io](https://crate.io)
for more info.

Crate.io published a [PDO](https://github.com/crate/crate-pdo) and
[DBAL](https://github.com/crate/crate-dbal) driver for easy access to
the crate DB server.
Laravel-crate.io project uses those adapters when connecting to Crate DB.

### Project status

**!!!! WARNING:** There's a breaking change in version 11. See changelog below.

Laravel-crate.io is used in our internal projects. We did a bunch of
unit tests and the driver seems ok. We use id as a caching layer in front
of our DB servers. Crate is insanely fast (Elasticsearch) and offloads our
DB servers a lot.

**if you find any bugs, please open an issue ticket**.

### Installation

Laravel 5.5, 6, 7, 8, 9, 10 and 11 are supported.

> [!NOTE]
> This package passes all it's tests and my own basic validation.
> It lacks production battle-testing as our systems run on earlier 
> Laravel versions (and I don't use them for this other tests any more).
> Please test thoroughly in your environment before production use, specially
> if you're using Laravel 11+. Also see notes below for version info.

Add a require to your composer.json :

```bash
    composer require ratkor/laravel-crate.io
```

> [!NOTE]
> Allways use tagged version in composer.json. Latest (master) version of laravel-crate.io
> works only with latest laravel. Latest laravel has a bunch of different function
> parameters in query parsing and you'll get parse errors if you'll try to use master
> branch with older laravel.

Laravel 5.5+ uses Package Auto-Discovery, so doesn't require you to manually add the ServiceProvider.

You'll have to install crate.io server, of course. See installation
instructions on their site.

#### PHP

Version 12 supports PHP 8.1.

Version 10.1 supports PHP 8.<br>
We run tests on ubuntu with PHP 8.0, crate-dbal 3 and phpunit 9.5. All tests passed ok.

Versions tagged with 9 and higher require **php 7.2.5**!

### Configuration

Open `config/database.php` and add new crate database connection (with
config values to match your setup):

```php
'crate' => array(
    'driver'   => 'crate',
    'host'     => 'localhost',
    'database' => 'doc',
    'port'     => 4200,
),
```

Next, change default database connection to `"crate"`.

```php
'default' => 'crate',
```

#### Configuration for HTTP Basic Auth

This driver supports standard CrateDB auth methods `trust` and `password` via
HTTP Basic Auth. To enable this, edit the `crate` config you created in the
previous step and add `username` and `password` with appropriate values for your system.

For more information on CrateDB auth methods, see CrateDB's documentation about [Authentication Methods](https://crate.io/docs/crate/reference/en/latest/admin/auth/methods.html).

```php
'crate' => array(
    'driver'      => 'crate',
    'host'        => 'localhost',
    'database'    => 'doc',
    'port'        => 4200,
    'username'    => 'MyUsername',
    'password'    => 'MyPassword',
),
```

#### Configuration for multiple hosts

This driver can handle conenctions to multiple crate hosts. To use them, write
`host` config parameter as a comma delimited list of hosts. Like:

```php
'crate' => array(
    'driver'   => 'crate',
    'host'     => 'localhost,10.0.0.1,10.0.0.2',
    'database' => 'doc',
    'port'     => 4200,
),
```

The DSN created in this case looks like:

```
'crate:localhost:4200,10.0.0.1:4200,10.0.0.2:4200'
```

If you need to specify different ports, add them to the `host` param like:

```php
    'host'     => 'localhost:4201,10.0.0.1:4300,10.0.0.2',
```

which will create a DSN like:

```
'crate:localhost:4201,10.0.0.1:4300,10.0.0.2:4200'
```

**Randomization**

`crate-pdo` takes [the first host](https://github.com/crate/crate-pdo/blob/master/src/Crate/PDO/PDO.php#L87)
from list of hosts. To overcome this we randomize all hosts
so that connections to multiple crate servers are distributed. If you don't want
randomization, add a `randomHosts` parameter and set it to `false`:

```php
'crate' => array(
    'driver'   => 'crate',
    'host'     => 'localhost,10.0.0.1,10.0.0.2',
    'database' => 'doc',
    'port'     => 4200,
    'randomHosts' => false,
),
```

### What works and what doesn't

Crate.io supports many of the SQL statements, but not all of them. Be sure to
take a look at their [site](https://crate.io/docs/stable/sql/index.html) if you're in doubt.

We're throwing an `RatkoR\Crate\NotImplementedException` for those statements that you
might wrongly try to use. We tried to cover all of them, but if we missed any
you'll get Exception from Crate DB.

Big things that are **not** supported are:

-   joins
-   subselects
-   [auto increments](https://crate.io/docs/stable/sql/ddl.html#constraints) - you'll have to manage those by yourself
-   whereBetween(s)
-   unique indexes
-   foreign keys (and relations)
-   dropping, renaming columns (adding fields works)
-   naming columns like _ id, _ version, \_ score - these are restricted, crate uses it [internally](https://crate.io/docs/stable/sql/ddl.html)

Crate specific stuff that were added are:

-   object type
-   array type
-   index off, index plain
-   fulltext indexes over single or multiple fields w/o analyzers
-   [table partitioning](https://crate.io/docs/crate/reference/en/latest/general/ddl/partitioned-tables.html)
-   [generated columns](https://crate.io/docs/crate/reference/en/latest/general/ddl/generated-columns.html#sql-ddl-generated-columns)

Also, `Article::truncate()` has been changed to silently use `delete from article`;

Note, that Crate.io does not support uppercase letters in table or schema names.
See this and other restrictions [here](https://crate.io/docs/stable/sql/ddl.html#naming-restrictions).

### Schema support

Migration and schema are supported. You can use `artisan migrate` commands
to create or drop tables.

Crate has only a subset of [field types](https://crate.io/docs/stable/sql/data_types.html)
(and some new ones), so choose appropriate.

Crate types:

-   boolean
-   string
-   numeric (integer, long, short, double, float, byte)
-   ip, geo_point (still have to implement these two)
-   timestamp
-   object
-   array

Some SQL types are silently linked to crate types. For example, `bigInteger` is
linked to `long`, `text, mediumtext, longtext, enum` are linked to `string`, ...

An example of schema in migration file would be:

```php
        Schema::create('article', function(Blueprint $table)
        {
            $table->integer('id');

$table->string('title')->index('plain');
            $table->mediumText('summary');
            $table->text('internal_Comment')->index('off');
            $table->text('body')->index('fulltext:english');
            $table->bigInteger('nb_views');
            $table->timestamp('published_on');

$table->arrayField('images','object as (id integer, title string)');
            $table->objectField('author','(dynamic) as (id integer, name string)');

$table->timestamps();

$table->primary('id');
        });
```

#### Blob tables

Creating (and dropping) blob tables is also supported. Blob tables don't have arbitrary
colums, just digest and last_moified. And even these are created automatically.

An example of create blob schema is:

```php
    Schema::createBlob('myblob');
```

_There is no need for the callback parameter (the second parameter in createBlob() which
defines fields). If you pass it it will be silently ignored._

To drop a table in schema do:

```php
    Schema::dropBlob('myblob');
```

#### Description of some SQL/Crate schema differences

**Fulltext index on a single field can be added as:**

```php
$table->index('field1','fulltext');
```

```php
$table->string('field1')->index('fulltext');
```

**Fulltext index on multiple fields:**

```php
$table->index(['field1','field2'],'fulltext');
```

**Fulltext index with english analyzer on multiple fields:**

```php
$table->index(['field1','field2'],'fulltext:english');
```

**Primary key on single field:**

```php
$table->primary('field1');
```

**Primary key on multiple fields:**

```php
$table->primary(['f_id','f2_id']);
```

**To not include a field in default index**

```php
$table->string('not_important_field')->index('off');
```

**A PLAIN index (the default index)**

```php
$table->string('field')->index('plain');
```

```php
$table->string('field')->index();
```

or just leave it out, crate will index it.

**To drop a table in migration scripts:**

```php
Schema::drop('article');
```

**To add an 'object' field use:**

```php
$table->objectField('field_name', 'object parameters');
```

where `object parameters` can be any parameters that crate expects for an object.
See their [documentation](https://crate.io/docs/stable/sql/data_types.html#object)
for objects. Examples would be:

```php
$table->objectField('my_object_1','as (f_date timestamp)');
$table->objectField('my_object_2','(dynamic) as (name string, birthday timestamp)');
```

**Add an 'array' field:**

Arrays are added with `->arrayField('name', 'array parameters')`. As is with
`object` type, `array paramters` can have any property that crate allows
for arrays. See their [documentation](https://crate.io/docs/stable/sql/data_types.html#array).
Examples for array of dynamic objects:

```php
$table->arrayField('f_array','object as (age integer, name string');
```

### Basic usage

With crate DB connection, you can do simple and even more complex queries.
Some examples are:

```php
$articles = DB::select('select * from article where id = ?', array(1));

$user = DB::table('user')->where('email','some@example.com')->first();

$users = DB::table('user')->get();
```

### Eloquent

To use Eloquent you'll need to use Crate Eloquent model.

```php
use RatkoR\Crate\Eloquent\Model AS Eloquent;

class Article extends Eloquent {}
```

You can use (almost) all eloquent goodies as with the original eloquent model.

To use different table name, use:

```php
protected $table = 'myArticles';
```

etc...

#### Eloquent model alias

Instead of adding

```php
use RatkoR\Crate\Eloquent\Model AS Eloquent;
```

to all your eloquent classes, you can add an alias to `alias` array in `config/app.php`:

```php
'CrateEloquent' => 'RatkoR\Crate\Eloquent\Model'
```

This will allow you to shorten the class definition to:

```php
use CrateEloquent;
class Article extends CrateEloquent {}
```

#### Eloquent usage

It can be used mostly the same as an original Laravel eloquent model.

##### Getting all articles:

```php
$articles = Article::all();
```

##### Getting by primary key:

```php
$article = Article::find(1);
```

##### Using where(s):

```php
$articles = Article::where('name','LIKE','Star%')->where('views','>',100)->get();
```

##### Using limits(s):

```php
$articles = Article::where('name','LIKE','Star%')->take(10)->get();
```

##### Using whereIn:

```php
$articles = Article::whereIn('id',[1,2,3])->get();
```

##### Using select for fields:

```php
$article = Article::select('id','name')->where('id',1)->first();
```

##### Using count:

```php
$nb = Article::where('views','>',100)->count();
```

##### Complex where(s):

```php
$articles = Article::where('id','=',3)->orWhere(function($query)
            {
                $query->where('title', 'Star Wars 7')
                      ->orWhere('title', 'none');
            })->get();
```

_etc..._

##### Inserting

```php
$new = Article::create([
    'id' => 1, // don't forget, there is no auto increment
    'title' => 'John Doe and friends',
    'summary' => '...',
    'array_of_strings' => ['one', 'two'],
    'object_field' => ['author' => 'Someone', 'title' => 'Editpr']
]);
```

##### Updating

```php
$article = Article::find(1);

$article->title = 'Brand new title';
$article->array_of_strings = ['tree', 'four'];
$article->object_field = ['author' => 'Someone Else', 'title' => 'Administrator'];
$article->save();
```

_Note_: when you update array or object field, whatever is in that field will be
replaced with whatever you give. You cannot append or change just one value.

```php
$article->object_field = ['crated_by' => 'Third Person'];
```

would _not_ append 'created_by' field to the fields that are already existing, but
would overwrite and leave only 'created_by' value in 'object_field'. To fix this,
do an update like:

```php
$newValues = $article->object_field;
$newValues['created_by'] = 'Third Person';

$article->object_field = $newValues;
```

##### Deleting

```php
$article = Article::find(1);
$article->delete();
```

### Changelog

#### Version 14

Support for laravel 11. We tested with laravel framework v11.46.2.

#### Version 13

Support for laravel 10.

#### Version 12

Support for laravel 9 (thanks to [Julian Martin](https://github.com/JulianMar))

#### Version 11

**BREAKING CHANGE**

Version 11 changes fetch method from `PDO::FETCH_ASSOC` to laravel's default
`PDO::FETCH_OBJ`.

#### Version 10.1

PHP 8 support added. All existing tests pass when run on machine with php 8
installed. No changes were needed in code (only composer.json was updated).

#### Version 10

Updated project to work with laravel 8.

**Note**: crate-pdo now allows saving of objects into string fields.

If you have a User object with `name` field set as string and if you do:

```
$foo = new stdClass();
$foo->bar = 'test';
User::create(['id'=>1,'name'=> $foo]);
```

it will do an insert with 'name' field set to object `{"bar": "test"}`. I don't know if it's a feature or a bug.. I'll leave it to the developers to use it or not.

`Connection::recordsHaveBeenModified()` is now properly called for all laravels that have it defined.

#### Version 9.1

Support for table patitioning and generated tables (pull #34).
Updated crate-dbal to version 2.0 (pull #33).

#### Version 9.0

Thanks to [Julian Martin](https://github.com/JulianMar) this project now works with
laravel 6.X. PHP 7.2 is now mandatory (it is mandatory for Laravel 6 also).

#### Version 8.0

Updated project to work with laravel 5.8
No new functionalities, only fixes compatibility issues with changes in L 5.8.

Reworked test cases to work with phpunit 8 and 9.

#### Version 7.0

Updated project to work with laravel 5.7
No new functionalities, only fixes compatibility issues with changes in L 5.7.

#### Version 6.0

Updated project to work with laravel 5.6

#### Version 5.0

Updated project to work with laravel 5.5. The new `5.0` version works
only with laravel 5.5.X.

#### Version 4.0

Updated and synced with laravel 5.4. The `runQueryCallback` method is
different in 5.4 as it was in 5.3. So you'll have to use 4.0 with laravel
5.4.

This is an internal change, you work with laravel and crate as you did
before. It's just a notice that 4.0 is not compatible with <5.3 laravels.

#### Version 3.1

##### Exceptions

Version 3.1. brings custom QueryException. This means that in case of
SQL errors you'll be able to see SQL and it's parameters even if some of
them are objects or arrays.

```
$foo = new stdClass();
$foo->bar = 'test';
User::create(['id'=>1,'name'=> $foo,'email'=>'user1@example.com']);
```

Throws:

```
QueryException:
SQLActionException [Validation failed for name: cannot cast {bar=test} to string] (SQL: insert into users (id, name, email) values (1, {"bar":"test"}, "user1@example.com"))
```

##### Connection

Connection to crate can take table prefix.
Add `prefix` key to your crate configuration to use it.

```php
'crate' => array(
    'driver'   => 'crate',
    'host'     => 'localhost',
    'database' => 'doc',
    'port'     => 4200,
    'prefix'   => 'sys',
),
```

##### New types

-   geoPoint
-   geoShape
-   ip

See [commit](https://github.com/RatkoR/laravel-crate.io/pull/10/commits/138289591e291aceb718e27a95123342be09b45b)
and official crate [docs](https://crate.io/a/geo-shapes-in-crate/)

### Tests

There are two kinds of tests:

-   Lexical tests
-   Data tests

### Lexical tests

Lexical tests check if SQL statements produced
by Query builder are semantically correct.

These tests are executed relatively fast. They check that all common
SQLs are unaffected by code changes.

### Data tests

Data tests connect to real Crate.io server and try to manage data there.
Selecting, inserting, updating, deleting queries, all are tested.

These tests take longer to finish. We found that queriying for a record immediatelly
after it has been inserted can produce negative results. Crate
needs some time between insert (or delete, or update) requests and
all next selects that query for this changes. So we have couple of
`sleep(1)` statements in test code.

Running data tests for the first time will probably fail as `migration`
table will not exist yet. Try rerunning test and it will proceed ok. Oh, and
you'll have to change `$table->increment('id')` to `$table->integer('id');` in
`DatabaseMigrationRepository::createRepository` or it will break as
increments are not supported.

Conenction properties for tests are in `tests/DataTests/Config/database.php`
file and can be changed for your setup.

Data tests will create:

-   `t_migration` table for test table migrations,
-   `t_users` table for some dummy user data.

laravel-crate.io

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README