Having worked on improving my suite of spiders, I wanted to expand the coverage to all the websites in the UK area of Openstreetmap and also incorporate more websites for a search engine I have in mind.

One idea to improve the speed and the number of websites I can index visit is by using serverless technology. This way I could create a server instance for each website spidering sessions and queue them all at once with AWS Lambda.

Using the AWS free tier, you can make 1 million request a month and 400,000 GB seconds of compute time per month, sounds like a lot!

So I started with a normal laravel install

composer create-project laravel/laravel aws-bref-serverless-lambda
cd aws-bref-serverless-lambda/
chmod -R 777 storage/

I have watched a number of videos to learn how to do this and this was the best to start with. It is made by @matthieunapoli

step 1- add laravel bridge

composer require bref/laravel-bridge --update-with-dependencies

step 2- create the serverless.yml file

php artisan vendor:publish --tag=serverless-config

step 3 - install serverless framework

It is made by twitter.com/goserverless/

npm install -g serverless

step 4 - configure serverless to access the aws account

by running the command 'serverless' I was able to configure serverless with the secret and access keys for AWS and deploy easily.

$ serverless

? No AWS credentials found, what credentials do you want to use? Local AWS Access Keys
? Do you have an AWS account? Yes

If your browser does not open automatically, please open this URL: 
https://console.aws.amazon.com/iam/home?region=us-east-1#/users$new?step=final&accessKey&userNames=serverless&permissionType=policies&policies=arn:aws:iam::aws:policy%2FAdministratorAccess

? In your AWS account, create an AWS user with access keys. Then press [Enter] to continue.
? AWS Access Key Id: 'key pasted here'
? AWS Secret Access Key: 'secret access key pasted here'

✔ AWS credentials saved on your machine at "~/.aws/credentials". Go there to change them at any time.

? Do you want to deploy now? Yes

Deploying laravel to stage dev (us-east-1)

✔ Service deployed to stack laravel-dev (197s)

endpoint: ANY - https://aaaaaaaaa.execute-api.us-east-1.amazonaws.com
functions:
  web: laravel-dev-web (30 MB)
  artisan: laravel-dev-artisan (30 MB)

What next?
Run these commands in the project directory:

serverless deploy    Deploy changes
serverless info      View deployed endpoints and resources
serverless invoke    Invoke deployed functions
serverless --help    Discover more commands

1 deprecation found: run 'serverless doctor' for more details

And it was live!

so i deleted it with

serverless remove

To deploy it again I ran

serverless deploy

and to run the php artisan cli command and it worked!

$ vendor/bin/bref cli laravel-dev-artisan inspire

  “ Happiness is not something readymade. It comes from your own actions. ”
  — Dalai Lama

In part 2 i will add some production level code to connect to a mysql database.

---

If you would like to contact me with this form on londinium.com, ilminster.net or via Twitter @andylondon