Typesense outputing the search results part 2

December 10, 2022

After the sucess of getting the basic search engine working. I looked at outputing the data as a neat webpage and also adding more fields and data to the json import. This is a second part to build...

Read this post ...

Emergency Debian Commands when the computer crashes to try and save it

December 9, 2022

I have an old laptop that often crashes when i use it. I was taking a backup of the files on it, but it seemed to hang a lot and became unresponsive so I wanted to write a blog post about how to...

Read this post ...

Setting Up A Typesense Website Search Engine

December 6, 2022

I wanted to setup a search engine for the urls and pages for my londinium.com site. I looked a options and found typesense which is an opensource solution and looks interesting. I installed it...

Read this post ...

Serverless Laravel with Bref using AWS Lambda Part 2

November 28, 2022

So my first idea is to get it working and up and running using my AWS ec2 debian instance which will call a lambda function and spider the website and save the data in the mysql db on the ec2 box....

Read this post ...

Serverless Laravel with Bref using AWS Lambda Part 1

November 27, 2022

Having worked on improving my suite of spiders, I wanted to expand the coverage to all the websites in the UK area of Openstreetmap and also incorporate more websites for a search engine I have in...

Read this post ...

Adding A Twitter Card Meta Tag Image

November 25, 2022

Today I have added a Twitter Card image for when I link to a post from Twitter. I had to write a blog post for the Jigsaw Static Site generator to include the image in the build. Here it is:

Read this post ...

Looking Up Domains for DNS Entries with PHP and Dig

November 23, 2022

One improvement I made today to speed up spidering website domains, is looking up the domain for a DNS-A entry. Using Curl or Guzzle is slowed down when the website has no entry in the DNS. So i...

Read this post ...

Using Screen and Links2 to run a PHP Web Spider and Detach the Terminal

November 22, 2022

I run a PHP based web spider to check the status of websites in londinium.com and was looking for a way to run it in the terminal. In the past I have run it using the refresh html header in a normal...

Read this post ...

Fixing Web Spider Errors with Guzzle Cookie Header Redirects and User Agent

November 21, 2022

Over the weekend, i ran a spider on the websites listed in the openstreetmap system. These are the error codes and the counts for these errors HTTP Error Count 401 13 402 6...

Read this post ...

Azure Free Tier Debian 11 and Ansible Installing a Lamp server

November 9, 2022

Today I got an email from Microsoft azure that a 30 day trial was ending and that I now had a 12 month fee basic tier level service, which is nice. I have set up a debian box to play with, just like...

Read this post ...