Browsed by
Tag: aws

How to offload cpu heavy code to lambda with Nodejs

How to offload cpu heavy code to lambda with Nodejs

Nodejs-2-562x309@2x-op

The title, says it all, but basically if you have a small service (in this case written with nodejs) running on a server with limited capabilities, you might run into problems if you want to do processor/memory heavy computations. For this can be a solution to offload the work to lambda, which can scale automatically, and you only have to pay for computation, so basically if you rarely need the computation, instead of renting a server you can do this and save heaps of money.

Talk is cheap, lets build it.

First of all if your dont already have you need the following:

Amazon AWS account

We will create the lambda function with this, and our extension of the application will run here.

Nodejs

Node.js is an open-source, cross-platform JavaScript runtime environment for developing a diverse variety of tools and applications.

You can get it here: download nodejs

NPM

Its for managing javascript packages, if you install nodejs, you will have this.

 

You have to log in to AWS and create a new lambda function:

new_lambda_function_button

You can choose the “Blank template” blueprint.

Name your function give it a description, a choose the Nodejs environment:

function_name_and_description

Insert the following code, it gives back a json when it recieves a request, if the json has an attribute named “data_to_transform” it gives back its value squared.

compute_heavy_task = function(data){
 return data*data;
}

exports.handler = (event, context, callback) => {
 context.succeed(
 {
 "lambda_function_name":"convert_stuff",
 "original_data":event.data_to_transform,
 "transformed_data":compute_heavy_task(event.data_to_transform)
 });
};

Create a role from a policy template and name it:

function_handler_and_role

review your settings:

review

Now you can test it, with the test settings and you’re supposed to get back a json like:

{ 
 lambda_function_name: 'convert_stuff',
 original_data: 500,
 transformed_data: 250000 
}

if your test request looks like:

{
        "data_to_transform":500
}

So in the second part we are going to create a nodejs app which has AWS auth credentials and can call the previously made function.

But first we have to create the credentials. Go to AWS console / IAM -> Users -> Add user.

welcome_to_iam   step1 step2

step3

step4

Save the credentials as a csv file. After clone the folowoing git repository npm install & run it with nodejs.

git clone https://github.com/yodeah/offload_to_lambda
cd offload_to_lambda
npm install

!! edit the conf file, with the neccesary data !!

node app.js

 

Nginx https load balancer with lets encrypt cert (On AWS)

Nginx https load balancer with lets encrypt cert (On AWS)

letsencrypt

Part 1: Create a working http load balancer

I’v decided to use amazon for hosting my (Ubuntu 14.04 trusty) server (t2.nano (still an overkill, anything with 256 mb ram is more than sufficent))

  1. you have to create a security profile which opens port 22 for ssh, 80 for http, and 443 for https.
  2. ssh into your server.
  3. Fetches the updates from the server, downloads nginx, apt-get update & upgrade, sudo apt-get install nginx
  4. Backup the config file, it is always considered a good practise to do. cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup
  5. Modify the http part of the config file to the code below, the 2 servers are the ones youre sending the load too (the connection to those is http). sudo nano /etc/nginx/nginx.conf
    http {
        upstream myapp1 {
            server google.com;
            server yahoo.com;
            }
    
        server {
            listen 80;
    
            location / {
                proxy_pass http://myapp1;
            }
        }  
    }
  6. Restart nginx (sudo service nginx restart), if everything is alright then you should have a working loadbalancer which responds with either something from google or yahoo. Congrats.

Part 2: Generating a cert & assigning it to nginx.

  1. Install the certbot script which helps you to get the cert quickly
    wget https://dl.eff.org/certbot-auto
    chmod a+x certbot-auto
    ./certbot-auto
    
  2. The prompt will guide you through, though it is recommended to turn off nginx while you do this, so you dont have anything listening on port 80, 443. After you have finished youll have your cert files in /etc/letsencrypt/live/yoururl
  3. Modify the nginx conf to use the cert files.
    http {
        upstream myapp1 {
            server google.com;
        }
    
            server{
                    listen 443 ssl;
                    server_name beta.daggersandsorcery.com www.daggersandsorcery$
    
                    ssl on;
                    ssl_certificate /etc/letsencrypt/live/beta.daggersandsorcery$
                    ssl_certificate_key /etc/letsencrypt/live/beta.daggersandsor$
    
    
                    location / {
                        proxy_pass http://myapp1;
                    }
            }
    }
    
  4. Restart nginx. Well done, it should be working for you.

Update:

The following script is doing the same thing that I’ve shown you in this article. With the addition of the automatic renewal process with crontab.

References: