After the sucess of getting the basic search engine working. I looked at outputing the data as a neat webpage and also adding more fields and data to the json import.

This is a second part to build on my first blog about Setting-Up-A-Typesense-Website-Search-Engine/

Typesense uses the jsonl format which is lines of json separated with the '\n' line break (note: the PHP_EOL function doesnt do this, it does '\r\n' for end of lines. This was my first gotcha!

The second Gotcha, was that the jsonl file had to match the names of the fields in the schema. I had an id field, that I didnt want to import, but that meant the import command ignored every row with extra fields in the jsonl file. So I removed the 2 fields I didn't need and indexed over 150,000 lines of json in a few minutes.

This is the file i used to import the json file.

<?php
require_once __DIR__ . '/vendor/autoload.php';

use Typesense\Client;

// setup the schema
$websiteSchema = [
  'name' => 'websites',
  'fields' => [
    ['name' => 'title', 'type' => 'string'],
    ['name' => 'description', 'type' => 'string', 'facet' => true],
    ['name' => 'website', 'type' => 'string'],
    ['name' => 'url', 'type' => 'string']
  ]
];

$client = new Client(
    [
      'api_key'         => 'API_KEY_HERE',
      'nodes'           => [
        [
          'host'     => 'localhost', // For Typesense Cloud use xxx.a1.typesense.net
          'port'     => '8108',      // For Typesense Cloud use 443
          'protocol' => 'http',      // For Typesense Cloud use https
        ],
      ],
      'connection_timeout_seconds' => 2,
    ]
);

// delete the schema
$client->collections['websites']->delete();

// create the new schema from above in typesense
$client->collections->create($websiteSchema);

// import json file of data
$websitesData = file_get_contents('websites.jsonl');
$client->collections['websites']->documents->import($websitesData, ['action' => 'create']);

// show the result
$result = $client->collections['websites']->retrieve();
var_dump($result);

This is the line that shows me the count of pages indexed and that it had worked!

  string(8) "websites"
  ["num_documents"]=>
  int(157788)

I made a basic controller to respond to queries in the url using laravel and a query as a GET string in the url like this http://localhost/search?q=chiswick&page=2 and here is the php code

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use Typesense\Client;

class SearchController extends Controller
{
    public function index(Request $request)
    {
        echo "Search for ";
        $q = $request['q'];
        $page = $request['page'];
        if (!isset($page)) {
            $page=1;
        }

        echo $q ."<br> page". $page;

        $client = new Client(
            [
              'api_key'         => 'API_KEY_HERE',
              'nodes'           => [
                [
                  'host'     => 'localhost', // For Typesense Cloud use xxx.a1.typesense.net
                  'port'     => '8108',      // For Typesense Cloud use 443
                  'protocol' => 'http',      // For Typesense Cloud use https
                ],
              ],
              'connection_timeout_seconds' => 2,
            ]
        );

        $searchParameters = [
            'q'         => $q,
            'query_by'  => 'description, title',
            'page'      => $page,
            'per_page'  => 50
          ];

        $results = $client->collections['websites']->documents->search($searchParameters);

        echo "<hr>found". $results['found'];

        foreach ($results['hits'] as $row) {
            // var_dump($row);
            echo $row['document']['url'];
            echo "<br>";
            echo $row['document']['website'];
            echo "<br>";
            echo $row['document']['title'];
            echo "<br>";
            echo $row['document']['description'];
            echo "<hr>";
        }
    }
}

and this creates a page like this:

And this is great for today. I am now going to add more columns of data from the database into the json file and import it.

Also going to look at making imports for other pages in the website, ranking these results and improving this all further.

Very exciting, as it hasnt taken very long to get a working version up and running.

If you would like to contact me with this form on londinium.com, ilminster.net or via Twitter @andylondon