Tag: memory

Memory profiler in Python

Posted on by SatyakiDe in api, Azure, cloud, code, Data Science, Pandas, Performance, Python

Today, I’ll be discussing a short but critical python topic. That is capturing the performance matrix by analyzing the memory profiling.

We’ll take any ordinary scripts & then use this package to analyze them.

But, before we start, why don’t we see the demo & then go through it?

Demo

Isn’t exciting? Let us understand in details.

For this, we’ve used the following package –

pip install memory-profiler

How you can run this?

All you have to do is to modify your existing python function & add this “profile” keyword. And this will open a brand new information shop for you.

#####################################################
#### Written By: SATYAKI DE                      ####
#### Written On: 22-Jul-2022                     ####
#### Modified On 30-Aug-2022                     ####
####                                             ####
#### Objective: This is the main calling         ####
#### python script that will invoke the          ####
#### clsReadForm class to initiate               ####
#### the reading capability in real-time         ####
#### & display text from a formatted forms.      ####
#####################################################

# We keep the setup code in a different class as shown below.
import clsReadForm as rf

from clsConfig import clsConfig as cf

import datetime
import logging

###############################################
###           Global Section                ###
###############################################
# Instantiating all the main class

x1 = rf.clsReadForm()

###############################################
###    End of Global Section                ###
###############################################
@profile
def main():
    try:
        # Other useful variables
        debugInd = 'Y'
        var = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
        var1 = datetime.datetime.now()

        print('Start Time: ', str(var))
        # End of useful variables

        # Initiating Log Class
        general_log_path = str(cf.conf['LOG_PATH'])

        # Enabling Logging Info
        logging.basicConfig(filename=general_log_path + 'readingForm.log', level=logging.INFO)

        print('Started extracting text from formatted forms!')

        # Execute all the pass
        r1 = x1.startProcess(debugInd, var)

        if (r1 == 0):
            print('Successfully extracted text from the formatted forms!')
        else:
            print('Failed to extract the text from the formatted forms!')

        var2 = datetime.datetime.now()

        c = var2 - var1
        minutes = c.total_seconds() / 60
        print('Total difference in minutes: ', str(minutes))

        print('End Time: ', str(var1))

    except Exception as e:
        x = str(e)
        print('Error: ', x)

if __name__ == "__main__":
    main()

Let us analyze the code. As you can see that, we’ve converted a normal python main function & mar it as @profile.

The next step is to run the following command –

python -m memory_profiler readingForm.py

This will trigger the script & it will collect all the memory information against individual lines & display it as shown in the demo.

I think this will give all the python developer a great insight about their quality of the code, which they have developed. To know more on this you can visit the following link.

I’ll bring some more exciting topic in the coming days from the Python verse. Please share & subscribe my post & let me know your feedback.

Till then, Happy Avenging! 🙂

Note: All the data & scenario posted here are representational data & scenarios & available over the internet & for educational purpose only. Some of the images (except my photo) that we’ve used are available over the net. We don’t claim the ownership of these images. There is an always room for improvement & especially the prediction quality.