Serverless design for LINE AI Chatbot

Chatbot is one of the interesting application in AI area, it creates opportunities for enterprise to serve customers only with very low cost or even generate new revenue.
In past few years, major Instant Messaging providers allow developers to hook their service. Means as long as you have existing simple message process and response system, you can quickly interact with all kind of message channel.

Normally, a software developer will start from build a system in a server box, no matter Linux or Windows. Recently, the server might be a VM in public cloud, no matter AWS, Azure, Linode or DigitalOcean. However, a serverless design model might be a better choice.

Why Serverless?

Firstly, a serverless system will be easy to scale in/out. It doesn't mean you can't scale in/out in traditional VM in public cloud or your own datacenter. It just means that all the Lambda, no matter which provider, is actually decouple from it development environment. Supposedly, you start from one Lambda function to a few thousands same Lambda function without consider "traditional question", for example: should I shutdown VM when not in peak our, should I do some script to check if current VMs are closed to overloading?

Secondly, a serverless system will be easy to plug-in which means during the design phase, developer will be forced to think de-couple functions in small modules (bricks). Developer will also be forced NOT to rely on specific environment, even though docker is one of the solution but purely Lambda function will create much better environment-free structure.

Furthermore, it will also help to define boundary of sub system and help the future maintenance.

The Design Concerns

(1) IM independent

LINE occupies a huge market in Taiwan, about more than 90% of mobile user has LINE account. The most incredible thing is many elder people who never touch Internet before have LINE accounts! However, this design won't use any LINE specific methods. We've try the same engine in Yahoo Messenger and it also works.

(2) AWS Lambda

-- (2.1) try NOT to use context

AWS Lambda has a standard invoke parameter (event, context), The event is actually the user input when invoke Lambda function. The context is what developer might need to understand the 'environment context'. The major design concern here is try NOT to use context when possible. Because this will make you hard to move out your lambda to other public cloud environment. If you really need to have ARN or identity, try to limit environment in just one Lambda.

-- (2.2) async invoke

AWS Lambda could be invoked in 3 types: Event, RequestResponse, DryRun. The "Event" is actually asynchronous call. For any IM message receiver Lambda, you should  keep that Lambda as simple as possible to response IM webhook. Put other things via "Event" Lambda. Because most of IM provider (LINE, fb) ask a very short timeout in IM webhook. DO NOT just put http webhook and response to IM a synchronous call stack

Of course, see detail from AWS document: here.

-- (2.3) timeout/memory

AWS lambda allow to config timeout and memory size. AWS CloudWatch could see a Lambda's resource consuming. It is fine to use larger memory or setup a longer running time but developer should know WHY.

-- (2.4) quick testing

It is necessary to have your own developer server for test your Lambda function and trigger a deployment script to upload to AWS. If you didn't actually use "context", it will be very simple to have a quick test in every Lambda handler.

# in the end of your Lambda python script.
if __name__ == '__main__':

    event = {'param1':test'}
    lambda_handler(event,None)

Of course developers need other framework (unittest).

-- (2.5) deployment

As always, from a developer should have a semi-automatic way to do deployment. This is a very simple deployment script to (a) zip python files (b) upload to S3 (c) create lambda function (d) config function using S3 zip file.

(a) zip lottery.zip -r lambda_lottery.py lottery60.py
(b) aws --profile ailine s3 cp lottery.zip s3://bucket/
(c) aws --profile ailine lambda create-function --function-name lottery --runtime py
thon3.6 --role "arn:aws:" --handler lambda_lott
ery.lambda_handler --timeout 10 --code "S3Bucket=bucket,S3Key=lottery.zip"
(d) aws --profile ailine lambda update-function-code --function-name lottery --s3-bu
cket bucket --s3-key lottery.zip

-- (2.6) scheduled (cron) Lambda

Chatbot might need to do scheduled task to response to user, maybe send a regular morning call. To trigger a scheduled Lambda might be one of the major cloud-provider-dependent thing we have in Chatbot design.

(3) AWS API Gateway

AWS API Gateway is another major cloud-provider-dependent things, however, it is not hard to use other provider or have our own lab testing environment. The major concerns of API Gateway are (a) should convert IM provider's http request to a given format: which becomes a Lambda input. (b) security concerns: how to make sure only IM provider's system could access this API Gateway

(4) AWS dynamodb

Chatbot uses dynamodb to store use information and also message log. It is also pretty easy to use local JSON formate nosql.

(5) AWS elasticsearch

Chatbot leverages AWS elasticsearch to store knowledge base. It is easy to setup a developer's elasticsearch server to do lab test before deployment. The real concerns in public cloud might be the future budget:)

(6) AWS S3

Chatbot still need some static content (html or js) and S3 is the most easy way to provide public static content. It is also the place to upload latest Lambda code.

The Implementation

See: github repository 

Take a look?

This chatbot could understand and speak only Tradition Chinese, since she is a Taiwanese robot:). You need to have LINE account to chat with her.

聊天機器人小姍的Line QR 

加小姍為好友