This site mainly deals with various use cases demonstrated using Python, Data Science, Cloud basics, SQL Server, Oracle, Teradata along with SQL & their implementation. Expecting yours active participation & time. This blog can be access from your TP, Tablet & mobile also. Please provide your feedback.
Today, I’m going to discuss a potential use case, where on many occasions, different teams need almost similar kinds of data through API. However, they are not identical. Creating a fresh API/Microservice after following-up with many processes will take significant time.
What if we can create an API in such a way so that we can get the response dynamically without needing to make another one. In this post, we’ll be demonstrating a similar approach.
I’ll be using open-source Covid-API, which will be useful for several posts starting from this one.
Let’s take two cases. One, where one service might need to access all the elements, there might be another, where some other service requires specific details.
Let’s explore the code base first –
init.py ( This native Python-based azure-function that will consume streaming data & dynamic API response. )
###########################################
#### Written By: SATYAKI DE ####
#### Written On: 06-Feb-2021 ####
#### Package Flask package needs to ####
#### install in order to run this ####
#### script. ####
#### ####
#### Objective: Main Calling scripts. ####
#### ####
#### However, to meet the functionality####
#### we've enhanced as per our logic. ####
###########################################
import logging
import json
import requests
import os
import pandas as p
import numpy as np
import azure.functions as func
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Dynamic-Covid-Status HTTP trigger function processed a request.')
try:
# Application Variable
url = os.environ['URL']
appType = os.environ['appType']
conType = os.environ['conType']
# API-Configuration
payload={}
headers = {
"Connection": conType,
"Content-Type": appType
}
# Validating input parameters
typeSel = req.params.get('typeSel')
if not typeSel:
try:
req_body = req.get_json()
except ValueError:
pass
else:
typeSel = req_body.get('typeSel')
typeVal = req.params.get('typeVal')
if not typeVal:
try:
req_body = req.get_json()
except ValueError:
pass
else:
typeVal = req_body.get('typeVal')
# Printing Key-Element Values
str1 = 'typeSel: ' + str(typeSel)
logging.info(str1)
str2 = 'typeVal: ' + str(typeVal)
logging.info(str2)
# End of API-Inputs
# Getting Covid data from the REST-API
response = requests.request("GET", url, headers=headers, data=payload)
ResJson = response.text
if typeSel == '*':
if typeVal != '':
# Converting it to Json
jdata = json.loads(ResJson)
df_ret = p.io.json.json_normalize(jdata)
df_ret.columns = df_ret.columns.map(lambda x: x.split(".")[-1])
rJson = df_ret.to_json(orient ='records')
return func.HttpResponse(rJson, status_code=200)
else:
x_stat = 'Failed'
x_msg = 'Important information is missing for all values!'
rJson = {
"status": x_stat,
"details": x_msg
}
xval = json.dumps(rJson)
return func.HttpResponse(xval, status_code=200)
elif typeSel == 'Cols':
if typeVal != '':
# Converting it to Json
jdata = json.loads(ResJson)
df_ret = p.io.json.json_normalize(jdata)
df_ret.columns = df_ret.columns.map(lambda x: x.split(".")[-1])
# Fetching for the selected columns
# Extracting the columns from the list
lstHead = []
listX = typeVal.split (",")
for i in listX:
lstHead.append(str(i).strip())
str3 = 'Main List: ' + str(lstHead)
logging.info(str3)
slice_df = df_ret[np.intersect1d(df_ret.columns, lstHead)]
rJson = slice_df.to_json(orient ='records')
return func.HttpResponse(rJson, status_code=200)
else:
x_stat = 'Failed'
x_msg = 'Important information is missing for selected values!'
rJson = {
"status": x_stat,
"details": x_msg
}
xval = json.dumps(rJson)
return func.HttpResponse(xval, status_code=200)
else:
x_stat = 'Failed'
x_msg = 'Important information is missing for typeSel!'
rJson = {
"status": x_stat,
"details": x_msg
}
xval = json.dumps(rJson)
return func.HttpResponse(xval, status_code=200)
except Exception as e:
x_msg = str(e)
x_stat = 'Failed'
rJson = {
"status": x_stat,
"details": x_msg
}
xval = json.dumps(rJson)
return func.HttpResponse(xval, status_code=200)
In the above lines, we’re converting the response & organizing it to a pandas dataframe before converting the response to JSON.
# Fetching for the selected columns
# Extracting the columns from the list
lstHead = []
listX = typeVal.split (",")
for i in listX:
lstHead.append(str(i).strip())
str3 = 'Main List: ' + str(lstHead)
logging.info(str3)
#slice_df = df_ret[df_ret.columns.intersection(lstHead)]
slice_df = df_ret[np.intersect1d(df_ret.columns, lstHead)]
For the second case, the above additional logic will play a significant part. Based on the supplied input in the typeVal attribute, this time, the new response will display accordingly.
Let’s see how it looks –
Azure function in Visual Studio Code
<p value="<amp-fit-text layout="fixed-height" min-font-size="6" max-font-size="72" height="80">Let's test it using Postman -Let’s test it using Postman –
Case 1 (For all the columns):
For all elements
And, the formatted output is as follows –
Formatted output for all elements
Case 2 (For selected columns):
For selected elements
<p value="<amp-fit-text layout="fixed-height" min-font-size="6" max-font-size="72" height="80">And, the formatted output is as follows -And, the formatted output is as follows –
Formatted output ofSelected element case
You can find the code in the Github using the following link.
So, finally, we have done it.
I’ll bring some more exciting topic in the coming days from the Python verse.
Till then, Happy Avenging! 😀
Note: All the data & scenario posted here are representational data & scenarios & available over the internet & for educational purpose only.
Today, I’ll be referring to two of my old posts & enhance one of them to create a basic online check-in app by integrating with Python. The best thing is – “All the components that will be in use already built using native Python code.”
And, probably, this will be the perfect tribute to growing Python fan-followers, who make this thing a reality.
We’ll be using the following API, which I shared in my previous post.
However, note that – to create any API environment better, one needs to use event-hub-based design & publish & consume events from there, instead of directly calling one microservice/API from another.
We need to use the following packages –
pip install briefcase
pip install requests
Apart from it you need to use JSON library as well.
The following steps are required to create this online check-in application –
STEP -1:
mkdir online_checkin
cd online_checkin
STEP -2:
python3 -m venv env
source env/bin/activate
STEP -3:
You need to install the desired packages mentioned above.
It should look something like –
Setting-up the environment
STEP -4:
Now, we have to execute the following command to initiate the project –
briefcase new
This will prompt to fill-up a set of inputs to configure the project as shown in the below screenshot –
Creation of a Mobile-App ProjectCreation of Mobile-App Project – Continuation
To check whether all the settings correctly captured or not, one can issue the following command –
briefcase dev
<p value="<amp-fit-text layout="fixed-height" min-font-size="6" max-font-size="72" height="80">If all the settings are correct, then a blank canvas iOS app will launch using the native Python code, which should look -If all the settings are correct, then a blank canvas iOS app will launch using the native Python code, which should look –
<p value="<amp-fit-text layout="fixed-height" min-font-size="6" max-font-size="72" height="80">
Empty iOS App launch
The above command will generate a series of directories & template python scripts.
Now, we are going to modify the app.py generated as part of the initial project creation.
app.py ( This iOS app will invoke online check-in API to receive the inputs & confirm the status based on the inputs selected by the passengers. )
################################################
#### ####
#### Written By: SATYAKI DE ####
#### Written On: 24-Nov-2020 ####
#### Briefcase, Toga, json, requests needs ####
#### to install to run this package. ####
#### ####
#### Objective: This script will create a ####
#### native I/OS App using native Python. ####
#### ####
################################################
"""
Calling Azure Microservice from Native Mobile App
"""
import toga
from toga.style import Pack
from toga.style.pack import COLUMN, ROW
import requests
import json
class online_checkin(toga.App):
def startup(self):
"""
Construct and show the Toga application.
Usually, you would add your application to a main content box.
We then create a main window (with a name matching the app), and
show the main window.
"""
main_box = toga.Box(style=Pack(direction=COLUMN))
# Adding Individual Layout details
name_label = toga.Label("Full Name", style=Pack(padding=(0, 5)))
self.name_input = toga.TextInput(style=Pack(flex=1))
name_box = toga.Box(style=Pack(direction=ROW, padding=5))
name_box.add(name_label)
name_box.add(self.name_input)
mobile_label = toga.Label("Mobile", style=Pack(padding=(0, 5)))
self.mobile_input = toga.TextInput(style=Pack(flex=1))
mobile_box = toga.Box(style=Pack(direction=ROW, padding=5))
mobile_box.add(mobile_label)
mobile_box.add(self.mobile_input)
email_label = toga.Label("Email", style=Pack(padding=(0, 5)))
self.email_input = toga.TextInput(style=Pack(flex=1))
email_box = toga.Box(style=Pack(direction=ROW, padding=5))
email_box.add(email_label)
email_box.add(self.email_input)
source_label = toga.Label("Source Airport", style=Pack(padding=(0, 5)))
self.source_input = toga.TextInput(style=Pack(flex=1))
source_box = toga.Box(style=Pack(direction=ROW, padding=5))
source_box.add(source_label)
source_box.add(self.source_input)
destination_label = toga.Label("Destination Airport", style=Pack(padding=(0, 5)))
self.destination_input = toga.TextInput(style=Pack(flex=1))
destination_box = toga.Box(style=Pack(direction=ROW, padding=5))
destination_box.add(destination_label)
destination_box.add(self.destination_input)
boardingclass_label = toga.Label("Boarding Class", style=Pack(padding=(0, 5)))
self.boardingclass_input = toga.TextInput(style=Pack(flex=1))
boardingclass_box = toga.Box(style=Pack(direction=ROW, padding=5))
boardingclass_box.add(boardingclass_label)
boardingclass_box.add(self.boardingclass_input)
preferredSeatNo_label = toga.Label("Preferred Seat", style=Pack(padding=(0, 5)))
self.preferredSeatNo_input = toga.TextInput(style=Pack(flex=1))
preferredSeatNo_box = toga.Box(style=Pack(direction=ROW, padding=5))
preferredSeatNo_box.add(preferredSeatNo_label)
preferredSeatNo_box.add(self.preferredSeatNo_input)
mealBreakfast_label = toga.Label("Breakfast Choice", style=Pack(padding=(0, 5)))
self.mealBreakfast_input = toga.TextInput(style=Pack(flex=1))
mealBreakfast_box = toga.Box(style=Pack(direction=ROW, padding=5))
mealBreakfast_box.add(mealBreakfast_label)
mealBreakfast_box.add(self.mealBreakfast_input)
mealLunch_label = toga.Label("Lunch Choice", style=Pack(padding=(0, 5)))
self.mealLunch_input = toga.TextInput(style=Pack(flex=1))
mealLunch_box = toga.Box(style=Pack(direction=ROW, padding=5))
mealLunch_box.add(mealLunch_label)
mealLunch_box.add(self.mealLunch_input)
mealDinner_label = toga.Label("Dinner Choice", style=Pack(padding=(0, 5)))
self.mealDinner_input = toga.TextInput(style=Pack(flex=1))
mealDinner_box = toga.Box(style=Pack(direction=ROW, padding=5))
mealDinner_box.add(mealDinner_label)
mealDinner_box.add(self.mealDinner_input)
passPort_label = toga.Label("Passport Details", style=Pack(padding=(0, 5)))
self.passPort_input = toga.TextInput(style=Pack(flex=1))
passPort_box = toga.Box(style=Pack(direction=ROW, padding=5))
passPort_box.add(passPort_label)
passPort_box.add(self.passPort_input)
localAddress_label = toga.Label("Local Address", style=Pack(padding=(0, 5)))
self.localAddress_input = toga.TextInput(style=Pack(flex=1))
localAddress_box = toga.Box(style=Pack(direction=ROW, padding=5))
localAddress_box.add(localAddress_label)
localAddress_box.add(self.localAddress_input)
bookingNo_label = toga.Label("Confirmed Booking No", style=Pack(padding=(0, 5)))
self.bookingNo_input = toga.TextInput(style=Pack(flex=1))
bookingNo_box = toga.Box(style=Pack(direction=ROW, padding=5))
bookingNo_box.add(bookingNo_label)
bookingNo_box.add(self.bookingNo_input)
# End Of Layout details
boardingStatus_box_label = toga.Label("Boarding Status", style=Pack(padding=(0, 5)))
self.boardingStatus_box_input = toga.MultilineTextInput(readonly=True, style=Pack(flex=1))
boardingStatus_box = toga.Box(style=Pack(direction=ROW, padding=5))
boardingStatus_box.add(boardingStatus_box_label)
boardingStatus_box.add(self.boardingStatus_box_input)
button = toga.Button("On-Board Now", on_press=self.onBoarding, style=Pack(padding=5))
# Adding all the visual layout in the main frame
main_box.add(name_box)
main_box.add(mobile_box)
main_box.add(email_box)
main_box.add(source_box)
main_box.add(destination_box)
main_box.add(boardingclass_box)
main_box.add(preferredSeatNo_box)
main_box.add(mealBreakfast_box)
main_box.add(mealLunch_box)
main_box.add(mealDinner_box)
main_box.add(passPort_box)
main_box.add(localAddress_box)
main_box.add(bookingNo_box)
main_box.add(button)
main_box.add(boardingStatus_box)
# End Of Main Frame
self.main_window = toga.MainWindow(title=self.formal_name)
self.main_window.content = main_box
self.main_window.show()
def onBoarding(self, widget):
BASE_URL = "https://xxxxxxxxxx.yyyyyyyy.net/api/getOnBoarding"
openmapapi_cache = "no-cache"
openmapapi_con = "keep-alive"
type = "application/json"
querystring = { "sourceLeg": self.source_input.value, "destinationLeg": self.destination_input.value,
"boardingClass": self.boardingclass_input.value, "preferredSeatNo": self.preferredSeatNo_input.value,
"travelPassport": self.passPort_input.value , "bookingNo": self.bookingNo_input.value,
"travelerEmail": self.email_input.value, "travelerMobile": self.mobile_input.value,
"mealBreakFast": self.mealBreakfast_input.value , "mealLunch": self.mealLunch_input.value,
"mealDinner": self.mealDinner_input.value, "localAddress": self.localAddress_input.value }
payload = json.dumps(querystring)
print('Input Payload: ')
print(payload)
headers = {
'content-type': type,
'Cache-Control': openmapapi_cache,
'Connection': openmapapi_con
}
response = requests.request("POST", BASE_URL, headers=headers, data=payload)
ResJson = response.text
#jdata = json.dumps(ResJson)
resp = json.loads(ResJson)
print('Response JSON:')
print(resp)
details = resp["description"].strip()
status = str(resp["status"]).strip()
sourceAirport = str(resp["sourceLeg"]).strip()
destinationAirport = str(resp["destinationLeg"]).strip()
boardingClass = str(resp["boardingClass"]).strip()
confirmedSeat = str(resp["confirmedSeatNo"]).strip()
try:
self.boardingStatus_box_input.value = f'Please find the update on your itenary -> Source - {sourceAirport} to Destination - {destinationAirport} - ' \
f'Boarding Class - {boardingClass} - Confirmed Seat - {confirmedSeat} and ' \
f'the summary is as follows - {details} - Status - {status}'
except ValueError:
self.boardingStatus_box_input.value = "Some technical issue occured. We are working on it."
def main():
return online_checkin()
From the above snippet, the program is building the input textbox on the iOS frame for the user input. This is more of a designing point of view. Other languages have sophisticated UI, which can generate similar kind of codes in the background. Unfortunately, Python still needs to grow in that aspect. However, if you think about the journey – it has progressed a lot.
# Adding all the visual layout in the main frame
main_box.add(name_box)
The above line finally stitches the widgets into the visual panel of the iOS app.
The above code first designs the button on the app & then it is tied with the “onBoarding” methods, which will invoke the API that we’ve already developed.
The above few lines framing an input JSON for our API & then invoke it & finally receive & parse the JSON to display the appropriate message on the iOS app.
The following image will show you the directory structure & also how the code should look. This will help you to map & understand the overall project.
Project Details
Let’s test our API –
Testing API through Postman
Now, we’re ready to test our iOS app.
You need to run the application locally as shown in the below image –
Running local iOS app using Native Python
As you can see that it is successfully able to fetch the API response & then parse it & populate the Boarding Status box marked in “Blue“.
Let us create the package for distribution by using the following command –
briefcase create
This will prompt a series of standard execution as shown below –
Building the package
Finally, you need to execute the following command to build the app –
briefcase build
Building the executed platform on the target OS
To find out whether your application ran successfully or not, one can use the following command –
briefcase run
This time it will run the application from this package & not from you local.
Now, the following commands help you to create & upload the package to iOS App Store.
briefcase create iOS
briefcase build iOS
You can refer the following official-site fore more information on creating the iOS package using briefcase.
Also, I find this is extremely useful for troubleshooting any issues here in this link.
So, finally, we have done it.
You will get the entire codebase from the following Github link.
I’ll bring some more exciting topic in the coming days from the Python verse.
Till then, Happy Avenging! 😀
Note:All the data & scenario posted here are representational data & scenarios & available over the internet & for educational purpose only.
Today, We will be discussing some of the better use-cases where we want to implement better solutions, which helps our entire Microservice/API integration painless.
As we drive towards more & more real-time applications such as mobile apps, IoT, and bots, we are engaging more microservice/API components in our technology stack. Moreover, We have witnessed that on many occasions, a full grown microservice environment leads to many unforeseen scenarios, which was not predicted or captured correctly.
To understand how a mesh-API architecture looks like, let us take one sample example for our case. Furthermore, what kind of problem we might face & how we can address.
Let us consider Airline online check-in as an example here.
There are following components are taking place irrespective of any airline ->
Filling-up primary information such as First Name, Last Name, Address, Contact details, Passport & Booking No.
We will have to choose a seat.
We will need to provide the input for a meal (However, this is optional as the domestic flight may or may not have this based on the distance & budget)
Finally, generations of Boarding-Pass.
If we decoupled our application here, we could create four inter-connected microservices to cater to our requirements. And this is shown in the given diagram –
Normal Microservice operations
From the above diagram, we can see that customer invokes a couple of related but independent services to finish the process of generating a Boarding-Pass. As per the diagram, initial requests reach the main Onboarding API & that results in cascading service calls & their successful response. In a developer’s term, this one we call it a “Happy Scenario!”
Nevertheless, developers do not always get this kind of rosy scenario. Sometimes, we encountered a scenario like this –
Cascading failure in microservice framework
Suppose we see an issue with our Boarding-pass generation service, which may encounter some network or any fatal issue, which leads to a chain of failure to its parent services. Moreover, the result will be disastrous.
How can we handle such scenarios? There are a few guidelines that we can explore.
Time-Out trigger while calling any service/s
Building Asynchronous service
Implementation of Circuit-Breaker
We can also use the standard workflow management or custom build workflow manager to improve that further (We will discuss this later in this blog).
We understood that our parent service should not wait for a longer time, where there is a significant delay from the child service. Otherwise, it will have a cascading effect & bring down the entire array of services and it.
However, the use of Circuit-Breaker is an advanced & better way to handle such scenarios. To keep this post small, We are not going to share all the codebase here. We will examine mainly two base codes.
Please find the current directory structure –
Current directory structure
Let us consider implementing without the Circuit-Breaker. We have created the following functions & a custom Python library deployed for this demo. However, We will not discuss anything in this library today. We will release it in PyPi & share the post the update & detail functionality here later. This library is a Python-based lightweight workflow manager, which will continuously monitor any failed service & when connected – it will trigger it when connected.
Use of Python-based Workflow Manager
From the above diagram, we can see that if we force “time-out” while calling our services, we can use this lightweight framework later whenever that service wake-up or establish a connection.
Please find the WhatsApp delivery through our microservice framework using this lightweight workflow framework.
Sending WhatsApp message using Python-based Workflow Manager
We have used Visual studio code editor for this entire development of azure functions. And, the screen should look something like this –
Development Editor
We’ll discuss one the following two services –
getSeatSelection
getBoardingPass
We will also discuss the first script twice – one is with the standard time-out approach & another with the circuit-breaker.
getSeatSelection( This Microservice receives the inputs from its parent microservice & confirm the preferred seat-no selected by the passengers. )
##############################################
#### Written By: SATYAKI DE ####
#### Written On: 05-Oct-2020 ####
#### Modified On 05-Oct-2020 ####
#### ####
#### Objective: Scripts to choose the ####
#### seat for the passenger. ####
##############################################
import logging
import json
import time
import os
import ssl
import requests
import azure.functions as func
def callService(inputJson, urlPost, retryNo, eventFlag, maxRetryNo, maxTimeOut):
# Invoking getBoardingPass API
try:
# Bypassing SSL Authentication
try:
_create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
# Legacy python that doesn't verify HTTPS certificates by default
pass
else:
# Handle target enviornment that doesn't support HTTPS verification
ssl._create_default_https_context = _create_unverified_https_context
# Providing the url bearer token
url = urlPost
json_data = inputJson
headers = {'content-type': "application/json", 'cache-control': "no-cache"}
# Failed case retry
retries = retryNo
success = False
return_description = 'N/A'
statusVal = 'Failed'
try:
while not success:
try:
# Getting response from web service
try:
strCheckMsg = 'Sending Payload - ' + str(retries)
logging.info(strCheckMsg)
response = requests.request("POST", url, data=json_data, headers=headers, verify=False, timeout=maxTimeOut)
resp_dict = json.loads(response.text)
statusVal = resp_dict['status']
return_description = resp_dict['description']
logging.info(statusVal)
except Exception as e:
x = str(e)
logging.info(x)
success = False
if (eventFlag == 'Boarding-Pass'):
str_R = "Boarding-Pass Json Response:: " + str(response.text)
else:
str_R = "Invalid flag options "
logging.info((str_R))
if len(response.text) > 80:
if str(response.status_code)[:1] == '2':
success = True
else:
wait = retries * 2
str_R1 = "retries Fail! Waiting " + str(wait) + " seconds and retrying!"
logging.info(str_R1)
time.sleep(wait)
retries += 1
# Checking maximum retries
if retries >= maxRetryNo:
success = True
raise Exception
else:
if str(response.status_code)[:1] == '2':
success = True
else:
# Checking maximum entries
if retries >= maxRetryNo:
success = True
raise Exception
retries += 1
except:
strVal = 'Retrying - ' + str(retries)
logging.info(strVal)
# Checking maximum entries
if retries >= maxRetryNo:
success = True
raise Exception
retries += 1
selection_flag = 'Y'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":statusVal,
"description":return_description,
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except (ConnectionError, TimeoutError, InterruptedError) as e:
str_R8 = "Response from Server: " + str(response)
logging.info(str_R8)
str_R9 = "Sending payload to Webservice!"
logging.info(str_R9)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Lost",
"description":"Timed-Out or Connection Error or any I/O issue preventing the process. We're working on this!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except ValueError as e:
x = str(e)
logging.info(x)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Failed",
"description":"Please check the internal parameters compatibility with it's data-type!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except Exception as e:
x = str(e)
logging.info(x)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Accepted",
"description":"Maximum rerties to the Boarding-Pass service is now reached. We're working offline to get that for you. Please rerty after 4 hours in case if you haven't received it or you can directly call customer care number!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except Exception as e:
x = str(e)
logging.info(x)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Failed",
"description":"DevOps Engineer is looking into this. Please try after some time!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Invoking Seat-Selection generation service.')
# Getting System Variable from local settings file
strVal = 'Parameter Recieved :' + os.environ['maxRetry']
logging.info(strVal)
# Capturing parameters from local settings
max_retries = int(os.environ['maxRetry'])
c_url = os.environ['cUrlBP']
# Printing input Payload
str_val = 'Input Payload:: ' + str(req.get_json())
logging.info(str_val)
strMainCheck = str(req.get_json())
# variable
x_status = 'Success'
if (strMainCheck != ''):
# Capturing individual elements
sourceLeg = req.params.get('sourceLeg')
destinationLeg = req.params.get('destinationLeg')
boardingClass = req.params.get('boardingClass')
preferredSeatNo = req.params.get('preferredSeatNo')
travelPassport = req.params.get('travelPassport')
bookingNo = req.params.get('bookingNo')
travelerEmail = req.params.get('travelerEmail')
travelerMobile = req.params.get('travelerMobile')
# Checking Individual Elements
if not sourceLeg:
try:
req_body = req.get_json()
except ValueError:
pass
else:
sourceLeg = req_body.get('sourceLeg')
if not destinationLeg:
try:
req_body = req.get_json()
except ValueError:
pass
else:
destinationLeg = req_body.get('destinationLeg')
if not boardingClass:
try:
req_body = req.get_json()
except ValueError:
pass
else:
boardingClass = req_body.get('boardingClass')
if not preferredSeatNo:
try:
req_body = req.get_json()
except ValueError:
pass
else:
preferredSeatNo = req_body.get('preferredSeatNo')
if not travelPassport:
try:
req_body = req.get_json()
except ValueError:
pass
else:
travelPassport = req_body.get('travelPassport')
if not bookingNo:
try:
req_body = req.get_json()
except ValueError:
pass
else:
bookingNo = req_body.get('bookingNo')
if not travelerEmail:
try:
req_body = req.get_json()
except ValueError:
pass
else:
travelerEmail = req_body.get('travelerEmail')
if not travelerMobile:
try:
req_body = req.get_json()
except ValueError:
pass
else:
travelerMobile = req_body.get('travelerMobile')
if (
(sourceLeg != '') & (destinationLeg != '') & (boardingClass != '') &
(preferredSeatNo != '') & (travelPassport != '') & (bookingNo != '') &
((travelerEmail != '') | (travelerMobile != ''))
):
x_description = "Seat-Selection Successfully Processed!"
# Preparing Payload for boarding-pass
json_str_boarding = {
"sourceLeg": sourceLeg,
"destinationLeg": destinationLeg,
"boardingClass": boardingClass,
"confirmedSeatNo": preferredSeatNo,
"travelPassport": travelPassport,
"bookingNo": bookingNo,
"travelerEmail": travelerEmail,
"travelerMobile": travelerMobile
}
jsonBoardingInput = json.dumps(json_str_boarding)
eventFlag = 'Boarding-Pass'
retryNoBase = 1
maxTimeOut = 2.5
# Invoking getBoardingPass API
microCallsBP = callService(jsonBoardingInput, c_url, retryNoBase, eventFlag, max_retries, maxTimeOut)
# Extracting seat_selection_flag
resp_dict = json.loads(microCallsBP)
x_status = resp_dict['status']
x_description = resp_dict['description']
# Capturing description value based on the logic
logging.info(x_description)
# Formatting return Payload
json_str = {
"description": x_description,
"status": x_status,
"sourceLeg": sourceLeg,
"destinationLeg": destinationLeg,
"boardingClass": boardingClass,
"confirmedSeatNo": preferredSeatNo,
"travelPassport": travelPassport,
"bookingNo": bookingNo,
"travelerEmail": travelerEmail,
"travelerMobile": travelerMobile
}
xval = json.dumps(json_str)
return func.HttpResponse(xval, status_code=200)
else:
json_str = {
"description": "Missing mandatory Email or Phone Number!",
"status": "Failed",
"sourceLeg": sourceLeg,
"destinationLeg": destinationLeg,
"boardingClass": boardingClass,
"confirmedSeatNo": preferredSeatNo,
"travelPassport": travelPassport,
"bookingNo": bookingNo,
"travelerEmail": travelerEmail,
"travelerMobile": travelerMobile
}
xval = json.dumps(json_str)
return func.HttpResponse(xval,status_code=200)
else:
json_str = {
"description": "Missing entire payload!",
"status": "Failed",
"sourceLeg": "N/A",
"destinationLeg": "N/A",
"boardingClass": "N/A",
"confirmedSeatNo": "N/A",
"travelPassport": "N/A",
"bookingNo": "N/A",
"travelerEmail": "N/A",
"travelerMobile": "N/A"
}
xval = json.dumps(json_str)
return func.HttpResponse(xval,status_code=200)
There are a few key snippets that we would like to discuss here.
# Getting response from web service
try:
strCheckMsg = 'Sending Payload - ' + str(retries)
logging.info(strCheckMsg)
response = requests.request("POST", url, data=json_data, headers=headers, verify=False, timeout=maxTimeOut)
resp_dict = json.loads(response.text)
statusVal = resp_dict['status']
return_description = resp_dict['description']
logging.info(statusVal)
except Exception as e:
x = str(e)
logging.info(x)
success = False
From the above snippet, we can see that we are using time-out based on our understanding of response time or the max SLA agreed upon for that particular service.
If we get the response, we are capturing the status as well as the description from its child service & capturing it in our log, which will look something like this –
Azure Monitor of Seat-Selection service
2. getBoardingPass ( This Microservice receives the input from its parent service & confirm the autogenerated boarding pass shared with their WhatsApp number.)
We tried to create a random wait time, which will produce a sudden network latency or mimicking potential network/time-out/connection issues. Also, we want to finish this thread to avoid any unwanted WhatsApp message trigger via Twilio-API.
And, when we want to test the parent Seat-Selection service through postman, it will show the following response –
Seat-Selection Microservice response
We must have noticed that since the boarding pass also captured the seat details; hence, we are addressing them together here.
For successful execution, we can see a similar message –
Seat-Selection service success status
However, if we want to handle this better, we can use “Circuit-Breaker.”
Let us understand what this is. Since our childhood, we all know that when there is an electricity fluctuation happens frequently, to prevent all the electrical appliances, we deploy this small device that will disconnect the power between our house’s primary grid with all the appliances at our home. It will prevent these devices from getting exposed.
The same concept has been implemented for Microservice architecture as well.
For Normal operation without any issue
The above diagram shows that using a circuit breaker will continue all its previous parent services in everyday activities without any issues.
Basic concept of the Circuit Breaker in Microservice
The above diagram shows that it will stop all the parent services if the child service is not responding based on the threshold count.
Circuit breaker with partially allow case
The above diagram shows that once there is a “STOP” event captured. It will wait for a few seconds before it will allow a few services to test whether the impacted service is ready to consume further requests or not. This way, it can control the threading bottleneck.
The next diagram will give us the distinct state of a “Circuit Breaker.”
States of a Circuit Breaker
The above diagram shows that using a circuit breaker will prevent all its previous parent services. It will maintain a threshold of failures & successful data points. It will allow services to invoke the impacted service either entirely, partially, or even completely block it depending on the situation.
Let us revise the code & review it together –
3. getSeatSelection ( This Microservice receives the inputs from its parent microservice & confirm the preferred seat-no selected by the passengers – using Circuit-Breaker. )
##############################################
#### Written By: SATYAKI DE ####
#### Written On: 05-Oct-2020 ####
#### Modified On 05-Oct-2020 ####
#### ####
#### Objective: Scripts to choose the ####
#### seat for the passenger. ####
##############################################
import logging
import json
import time
import os
import ssl
import requests
import azure.functions as func
from circuitbreaker import CircuitBreaker
class BoardingPassCircuitBreaker(CircuitBreaker):
FAILURE_THRESHOLD = 10
RECOVERY_TIMEOUT = 60
EXPECTED_EXCEPTION = TimeoutError
@BoardingPassCircuitBreaker()
def callService(inputJson, urlPost, retryNo, eventFlag, maxRetryNo, maxTimeOut):
# Invoking getBoardingPass API
try:
# Bypassing SSL Authentication
try:
_create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
# Legacy python that doesn't verify HTTPS certificates by default
pass
else:
# Handle target enviornment that doesn't support HTTPS verification
ssl._create_default_https_context = _create_unverified_https_context
# Providing the url bearer token
url = urlPost
json_data = inputJson
headers = {'content-type': "application/json", 'cache-control': "no-cache"}
# Failed case retry
retries = retryNo
success = False
return_description = 'N/A'
statusVal = 'Failed'
try:
while not success:
try:
# Getting response from web service
try:
strCheckMsg = 'Sending Payload - ' + str(retries)
logging.info(strCheckMsg)
response = requests.request("POST", url, data=json_data, headers=headers, verify=False, timeout=maxTimeOut)
resp_dict = json.loads(response.text)
statusVal = resp_dict['status']
return_description = resp_dict['description']
logging.info(statusVal)
except Exception as e:
x = str(e)
logging.info(x)
success = False
if (eventFlag == 'Boarding-Pass'):
str_R = "Boarding-Pass Json Response:: " + str(response.text)
else:
str_R = "Invalid flag options "
logging.info((str_R))
if len(response.text) > 80:
if str(response.status_code)[:1] == '2':
success = True
else:
wait = retries * 2
str_R1 = "retries Fail! Waiting " + str(wait) + " seconds and retrying!"
logging.info(str_R1)
time.sleep(wait)
retries += 1
# Checking maximum retries
if retries >= maxRetryNo:
success = True
raise Exception
else:
if str(response.status_code)[:1] == '2':
success = True
else:
# Checking maximum entries
if retries >= maxRetryNo:
success = True
raise Exception
retries += 1
except:
strVal = 'Retrying - ' + str(retries)
logging.info(strVal)
# Checking maximum entries
if retries >= maxRetryNo:
success = True
raise Exception
retries += 1
selection_flag = 'Y'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":statusVal,
"description":return_description,
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except (ConnectionError, TimeoutError, InterruptedError) as e:
str_R8 = "Response from Server: " + str(response)
logging.info(str_R8)
str_R9 = "Sending payload to Webservice!"
logging.info(str_R9)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Lost",
"description":"Timed-Out or Connection Error or any I/O issue preventing the process. We're working on this!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except ValueError as e:
x = str(e)
logging.info(x)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Failed",
"description":"Please check the internal parameters compatibility with it's data-type!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except Exception as e:
x = str(e)
logging.info(x)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Accepted",
"description":"Maximum rerties to the Boarding-Pass service is now reached. We're working offline to get that for you. Please rerty after 4 hours in case if you haven't received it or you can directly call customer care number!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except Exception as e:
x = str(e)
logging.info(x)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Failed",
"description":"DevOps Engineer is looking into this. Please try after some time!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Invoking Seat-Selection generation service.')
# Getting System Variable from local settings file
strVal = 'Parameter Recieved :' + os.environ['maxRetry']
logging.info(strVal)
# Capturing parameters from local settings
max_retries = int(os.environ['maxRetry'])
c_url = os.environ['cUrlBP']
# Printing input Payload
str_val = 'Input Payload:: ' + str(req.get_json())
logging.info(str_val)
strMainCheck = str(req.get_json())
# variable
x_status = 'Success'
if (strMainCheck != ''):
# Capturing individual elements
sourceLeg = req.params.get('sourceLeg')
destinationLeg = req.params.get('destinationLeg')
boardingClass = req.params.get('boardingClass')
preferredSeatNo = req.params.get('preferredSeatNo')
travelPassport = req.params.get('travelPassport')
bookingNo = req.params.get('bookingNo')
travelerEmail = req.params.get('travelerEmail')
travelerMobile = req.params.get('travelerMobile')
# Checking Individual Elements
if not sourceLeg:
try:
req_body = req.get_json()
except ValueError:
pass
else:
sourceLeg = req_body.get('sourceLeg')
if not destinationLeg:
try:
req_body = req.get_json()
except ValueError:
pass
else:
destinationLeg = req_body.get('destinationLeg')
if not boardingClass:
try:
req_body = req.get_json()
except ValueError:
pass
else:
boardingClass = req_body.get('boardingClass')
if not preferredSeatNo:
try:
req_body = req.get_json()
except ValueError:
pass
else:
preferredSeatNo = req_body.get('preferredSeatNo')
if not travelPassport:
try:
req_body = req.get_json()
except ValueError:
pass
else:
travelPassport = req_body.get('travelPassport')
if not bookingNo:
try:
req_body = req.get_json()
except ValueError:
pass
else:
bookingNo = req_body.get('bookingNo')
if not travelerEmail:
try:
req_body = req.get_json()
except ValueError:
pass
else:
travelerEmail = req_body.get('travelerEmail')
if not travelerMobile:
try:
req_body = req.get_json()
except ValueError:
pass
else:
travelerMobile = req_body.get('travelerMobile')
if (
(sourceLeg != '') & (destinationLeg != '') & (boardingClass != '') &
(preferredSeatNo != '') & (travelPassport != '') & (bookingNo != '') &
((travelerEmail != '') | (travelerMobile != ''))
):
x_description = "Seat-Selection Successfully Processed!"
# Preparing Payload for boarding-pass
json_str_boarding = {
"sourceLeg": sourceLeg,
"destinationLeg": destinationLeg,
"boardingClass": boardingClass,
"confirmedSeatNo": preferredSeatNo,
"travelPassport": travelPassport,
"bookingNo": bookingNo,
"travelerEmail": travelerEmail,
"travelerMobile": travelerMobile
}
jsonBoardingInput = json.dumps(json_str_boarding)
eventFlag = 'Boarding-Pass'
retryNoBase = 1
maxTimeOut = 2.5
# Invoking getBoardingPass API
microCallsBP = callService(jsonBoardingInput, c_url, retryNoBase, eventFlag, max_retries, maxTimeOut)
# Extracting seat_selection_flag
resp_dict = json.loads(microCallsBP)
x_status = resp_dict['status']
x_description = resp_dict['description']
# Capturing description value based on the logic
logging.info(x_description)
# Formatting return Payload
json_str = {
"description": x_description,
"status": x_status,
"sourceLeg": sourceLeg,
"destinationLeg": destinationLeg,
"boardingClass": boardingClass,
"confirmedSeatNo": preferredSeatNo,
"travelPassport": travelPassport,
"bookingNo": bookingNo,
"travelerEmail": travelerEmail,
"travelerMobile": travelerMobile
}
xval = json.dumps(json_str)
return func.HttpResponse(xval, status_code=200)
else:
json_str = {
"description": "Missing mandatory Email or Phone Number!",
"status": "Failed",
"sourceLeg": sourceLeg,
"destinationLeg": destinationLeg,
"boardingClass": boardingClass,
"confirmedSeatNo": preferredSeatNo,
"travelPassport": travelPassport,
"bookingNo": bookingNo,
"travelerEmail": travelerEmail,
"travelerMobile": travelerMobile
}
xval = json.dumps(json_str)
return func.HttpResponse(xval,status_code=200)
else:
json_str = {
"description": "Missing entire payload!",
"status": "Failed",
"sourceLeg": "N/A",
"destinationLeg": "N/A",
"boardingClass": "N/A",
"confirmedSeatNo": "N/A",
"travelPassport": "N/A",
"bookingNo": "N/A",
"travelerEmail": "N/A",
"travelerMobile": "N/A"
}
xval = json.dumps(json_str)
return func.HttpResponse(xval,status_code=200)
Let us review the key snippets –
from circuitbreaker import CircuitBreaker
class BoardingPassCircuitBreaker(CircuitBreaker):
FAILURE_THRESHOLD = 10
RECOVERY_TIMEOUT = 60
EXPECTED_EXCEPTION = TimeoutError
@BoardingPassCircuitBreaker()
def callService(inputJson, urlPost, retryNo, eventFlag, maxRetryNo, maxTimeOut):
# Invoking getBoardingPass API
try:
<Code>
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":statusVal,
"description":return_description,
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
except Exception as e:
x = str(e)
logging.info(x)
selection_flag = 'N'
# Forming return JSON
jsonRet = {
"eventFlag": eventFlag,
"status":"Failed",
"description":"DevOps Engineer is looking into this. Please try after some time!",
"computeFlag":selection_flag
}
xval = json.dumps(jsonRet)
return xval
We have put the microservice call into a function & then mark it as a circuit-breaker method. Also, we have enforced some custom settings in our main circuit-breaker class as well.
Moreover, here are some of our main API sample outputs, i.e., “getOnBoarding,” depending on the child-services’ availability.
Please click the above image gallery to see all the combinations of responses.
Finally, here is the WhatsApp message if that call successful. We have used Twilio API to exchange the text.
Boarding Pass delivery to Passenger WhatsApp No
Please find the dependent package from requirements.txt ->
You must be logged in to post a comment.