Real-time video summary assistance App – Part 1

Today, we’ll discuss another topic in our two-part series. We will understand the importance of the MCP protocol for communicating between agents.

This will be an in-depth highly technical as well as depicting using easy-to-understand visuals.

But, before that, let us understand the demo first.

Isn’t it exciting?


Let us first understand in easy language about the MCP protocol.

MCP (Multi-Agent Communication Protocol) is a custom message exchange system that facilitates structured and scalable communication among multiple AI agents operating within an application. These agents collaborate asynchronously or in real-time to complete complex tasks by sharing results, context, and commands through a common messaging layer.

How MCP Protocol Helps:

FeatureBenefit
Agent-Oriented ArchitectureEach agent handles a focused task, improving modularity and scalability.
Event-Driven Message PassingAgents communicate based on triggers, not polling—leading to faster and efficient responses.
Structured Communication FormatAll messages follow a standard format (e.g., JSON) with metadata for sender, recipient, type, and payload.
State PreservationAgents maintain context across messages using memory (e.g., ConversationBufferMemory) to ensure coherence.

How It Works (Step-by-Step):

  • 📥 User uploads or streams a video.
  • 🧑‍💻 MCP Protocol triggers the Transcription Agent to start converting audio into text.
  • 🌐 Translation Agent receives this text (if a different language is needed).
  • 🧾 Summarization Agent receives the translated or original transcript and generates a concise summary.
  • 📚 Research Agent checks for references or terminology used in the video.
  • 📄 Documentation Agent compiles the output into a structured report.
  • 🔁 All communication between agents flows through MCP, ensuring consistent message delivery and coordination.

Now, let us understand the solution that we intend to implement for our solutions:

This app provides live summarization and contextual insights from videos such as webinars, interviews, or YouTube recordings using multiple cooperating AI agents. These agents may include:

  • Transcription Agent: Converts spoken words to text.
  • Translation Agent: Translates text to different languages (if needed).
  • Summarization Agent: Generates concise summaries.
  • Research Agent: Finds background or supplementary data related to the discussion.
  • Documentation Agent: Converts outputs into structured reports or learning materials.

We need to understand one more thing before deep diving into the code. Part of your conversation may be mixed, like part Hindi & part English. So, in that case, it will break the sentences into chunks & then convert all of them into the same language. Hence, the following rules are applied while translating the sentences –


Now, we will go through the basic frame of the system & try to understand how it fits all the principles that we discussed above for this particular solution mapped against the specific technology –

  1. Documentation Agent built with the LangChain framework
  2. Research Agent built with the AutoGen framework
  3. MCP Broker for seamless communication between agents

Let us understand from the given picture the flow of the process that our app is trying to implement –


Great! So, now, we’ll focus on some of the key Python scripts & go through their key features.

But, before that, we share the group of scripts that belong to specific tasks.

  • clsMCPMessage.py
  • clsMCPBroker.py
  • clsYouTubeVideoProcessor.py
  • clsLanguageDetector.py
  • clsTranslationAgent.py
  • clsTranslationService.py
  • clsDocumentationAgent.py
  • clsResearchAgent.py

Now, we’ll review some of the script in this post, along with the next post, as a continuation from this post.

class clsMCPMessage(BaseModel):
    """Message format for MCP protocol"""
    id: str = Field(default_factory=lambda: str(uuid.uuid4()))
    timestamp: float = Field(default_factory=time.time)
    sender: str
    receiver: str
    message_type: str  # "request", "response", "notification"
    content: Dict[str, Any]
    reply_to: Optional[str] = None
    conversation_id: str
    metadata: Dict[str, Any] = {}
    
class clsMCPBroker:
    """Message broker for MCP protocol communication between agents"""
    
    def __init__(self):
        self.message_queues: Dict[str, queue.Queue] = {}
        self.subscribers: Dict[str, List[str]] = {}
        self.conversation_history: Dict[str, List[clsMCPMessage]] = {}
    
    def register_agent(self, agent_id: str) -> None:
        """Register an agent with the broker"""
        if agent_id not in self.message_queues:
            self.message_queues[agent_id] = queue.Queue()
            self.subscribers[agent_id] = []
    
    def subscribe(self, subscriber_id: str, publisher_id: str) -> None:
        """Subscribe an agent to messages from another agent"""
        if publisher_id in self.subscribers:
            if subscriber_id not in self.subscribers[publisher_id]:
                self.subscribers[publisher_id].append(subscriber_id)
    
    def publish(self, message: clsMCPMessage) -> None:
        """Publish a message to its intended receiver"""
        # Store in conversation history
        if message.conversation_id not in self.conversation_history:
            self.conversation_history[message.conversation_id] = []
        self.conversation_history[message.conversation_id].append(message)
        
        # Deliver to direct receiver
        if message.receiver in self.message_queues:
            self.message_queues[message.receiver].put(message)
        
        # Deliver to subscribers of the sender
        for subscriber in self.subscribers.get(message.sender, []):
            if subscriber != message.receiver:  # Avoid duplicates
                self.message_queues[subscriber].put(message)
    
    def get_message(self, agent_id: str, timeout: Optional[float] = None) -> Optional[clsMCPMessage]:
        """Get a message for the specified agent"""
        try:
            return self.message_queues[agent_id].get(timeout=timeout)
        except (queue.Empty, KeyError):
            return None
    
    def get_conversation_history(self, conversation_id: str) -> List[clsMCPMessage]:
        """Get the history of a conversation"""
        return self.conversation_history.get(conversation_id, [])

Imagine a system where different virtual agents (like robots or apps) need to talk to each other. To do that, they send messages back and forth—kind of like emails or text messages. This code is responsible for:

  • Making sure those messages are properly written (like filling out all parts of a form).
  • Making sure messages are delivered to the right people.
  • Keeping a record of conversations so you can go back and review what was said.

This part (clsMCPMessage) is like a template or a form that every message needs to follow. Each message has:

  • ID: A unique number so every message is different (like a serial number).
  • Time Sent: When the message was created.
  • Sender & Receiver: Who sent the message and who is supposed to receive it.
  • Type of Message: Is it a request, a response, or just a notification?
  • Content: The actual information or question the message is about.
  • Reply To: If this message is answering another one, this tells which one.
  • Conversation ID: So we know which group of messages belongs to the same conversation.
  • Extra Info (Metadata): Any other small details that might help explain the message.

This (clsMCPBroker) is the system (or “post office”) that makes sure messages get to where they’re supposed to go. Here’s what it does:

1. Registering an Agent

  • Think of this like signing up a new user in the system.
  • Each agent gets their own personal mailbox (called a “message queue”) so others can send them messages.

2. Subscribing to Another Agent

  • If Agent A wants to receive copies of messages from Agent B, they can “subscribe” to B.
  • This is like signing up for B’s newsletter—whenever B sends something, A gets a copy.

3. Sending a Message

  • When someone sends a message:
    • It is saved into a conversation history (like keeping emails in your inbox).
    • It is delivered to the main person it was meant for.
    • And, if anyone subscribed to the sender, they get a copy too—unless they’re already the main receiver (to avoid sending duplicates).

4. Receiving Messages

  • Each agent can check their personal mailbox to see if they got any new messages.
  • If there are no messages, they’ll either wait for some time or move on.

5. Viewing Past Conversations

  • You can look up all messages that were part of a specific conversation.
  • This is helpful for remembering what was said earlier.

In systems where many different smart tools or services need to work together and communicate, this kind of communication system makes sure everything is:

  • Organized
  • Delivered correctly
  • Easy to trace back when needed

So, in this post, we’ll finish it here. We’ll cover the rest of the post in the next post.

I’ll bring some more exciting topics in the coming days from the Python verse.

Till then, Happy Avenging!  🙂

Creating a local LLM Cluster Server using Apple Silicon GPU

Today, we’re going to discuss creating a local LLM server and then utilizing it to execute various popular LLM models. We will club the local Apple GPUs together via a new framework that binds all the available Apple Silicon devices into one big LLM server. This enables people to run many large models, which was otherwise not possible due to the lack of GPUs.

This is certainly a new way; One can create virtual computation layers by adding nodes to the resource pool, increasing the computation capacity.

Why not witness a small demo to energize ourselves –

Let us understand the scenario. I’ve one Mac Book Pro M4 & 2 Mac Mini Pro M4 (Base models). So, I want to add them & expose them as a cluster as follows –

As you can see, I’ve connected my MacBook Pro with both the Mac Mini using high-speed thunderbolt cables for better data transmissions. And, I’ll be using an open-source framework called “Exo” to create it.

Also, you can see that my total computing capacity is 53.11 TFlops, which is slightly more than the last category.

“Exo” is an open-source framework that helps you merge all your available devices into a large cluster of available resources. This extracts all the computing juice needed to handle complex tasks, including the big LLMs, which require very expensive GPU-based servers.

For more information on “Exo”, please refer to the following link.

In our previous diagram, we can see that the framework also offers endpoints.

  • One option is a local ChatGPT interface, where any question you ask will receive a response from models by combining all available computing power.
  • The other endpoint offers users a choice of any standard LLM API endpoint, which helps them integrate it into their solutions.

Let us see, how the devices are connected together –


To proceed with this, you need to have at least Python 3.12, Anaconda or Miniconda & Xcode installed in all of your machines. Also, you need to install some Apple-specific MLX packages or libraries to get the best performance.

Depending on your choice, you need to use the following link to download Anaconda or Miniconda.

You can download the following link to download the Python 3.12. However, I’ve used Python 3.13 on some machines & some machines, I’ve used Python 3.12. And it worked without any problem.

Sometimes, after installing Anaconda or Miniconda, the environment may not implicitly be activated after successful installation. In that case, you may need to use the following commands in the terminal -> source ~/.bash_profile

To verify, whether the conda has been successfully installed & activated, you need to type the following command –

(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % conda --version
conda 24.11.3
(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 
(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 

Once you verify it. Now, we need to install the following supplemental packages in all the machines as –

satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 
satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 
satyaki_de@Satyakis-MacBook-Pro-Max Pandas % conda install anaconda::m4
Channels:
 - defaults
 - anaconda
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/anaconda3

  added / updated specs:
    - anaconda::m4


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    m4-1.4.18                  |       h1230e6a_1         202 KB  anaconda
    ------------------------------------------------------------
                                           Total:         202 KB

The following NEW packages will be INSTALLED:

  m4                 anaconda/osx-arm64::m4-1.4.18-h1230e6a_1 


Proceed ([y]/n)? y


Downloading and Extracting Packages:
                                                                                                                                                                                                                      
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Also, you can use this package to install in your machines –

(base) satyakidemini2@Satyakis-Mac-mini-2 exo % 
(base) satyakidemini2@Satyakis-Mac-mini-2 exo % pip install mlx
Collecting mlx
  Downloading mlx-0.23.2-cp312-cp312-macosx_14_0_arm64.whl.metadata (5.3 kB)
Downloading mlx-0.23.2-cp312-cp312-macosx_14_0_arm64.whl (27.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 27.6/27.6 MB 8.8 MB/s eta 0:00:00
Installing collected packages: mlx
Successfully installed mlx-0.23.2
(base) satyakidemini2@Satyakis-Mac-mini-2 exo % 
(base) satyakidemini2@Satyakis-Mac-mini-2 exo % 

Till now, we’ve installed all the important packages. Now, we need to setup the final “eco” framework in all the machines like our previous steps.

Now, we’ll first clone the “eco” framework by the following commands –

(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 
(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 
(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % git clone https://github.com/exo-explore/exo.git
Cloning into 'exo'...
remote: Enumerating objects: 9736, done.
remote: Counting objects: 100% (411/411), done.
remote: Compressing objects: 100% (148/148), done.
remote: Total 9736 (delta 333), reused 263 (delta 263), pack-reused 9325 (from 3)
Receiving objects: 100% (9736/9736), 12.18 MiB | 8.41 MiB/s, done.
Resolving deltas: 100% (5917/5917), done.
Updating files: 100% (178/178), done.
Filtering content: 100% (9/9), 3.16 MiB | 2.45 MiB/s, done.
(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 
(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % 

And, the content of the “Exo” folder should look like this –

total 28672
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 docs
-rwx------  1 satyaki_de  staff     1337 Mar  9 17:06 configure_mlx.sh
-rwx------  1 satyaki_de  staff    11107 Mar  9 17:06 README.md
-rwx------  1 satyaki_de  staff    35150 Mar  9 17:06 LICENSE
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 examples
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 exo
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 extra
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 scripts
-rwx------  1 satyaki_de  staff      390 Mar  9 17:06 install.sh
-rwx------  1 satyaki_de  staff      792 Mar  9 17:06 format.py
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 test
-rwx------  1 satyaki_de  staff     2476 Mar  9 17:06 setup.py
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:10 build
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:17 exo.egg-info

Similar commands need to fire to other devices. Here, I’m showing one Mac-Mini examples –

(base) satyakidemini2@Satyakis-Mac-mini-2 Pandas % 
(base) satyakidemini2@Satyakis-Mac-mini-2 Pandas % git clone https://github.com/exo-explore/exo.git
Cloning into 'exo'...
remote: Enumerating objects: 9736, done.
remote: Counting objects: 100% (424/424), done.
remote: Compressing objects: 100% (146/146), done.
remote: Total 9736 (delta 345), reused 278 (delta 278), pack-reused 9312 (from 4)
Receiving objects: 100% (9736/9736), 12.18 MiB | 6.37 MiB/s, done.
Resolving deltas: 100% (5920/5920), done.
(base) satyakidemini2@Satyakis-Mac-mini-2 Pandas % 

After that, I’ll execute the following sets of commands to install the framework –

(base) satyaki_de@Satyakis-MacBook-Pro-Max Pandas % cd exo
(base) satyaki_de@Satyakis-MacBook-Pro-Max exo % 
(base) satyaki_de@Satyakis-MacBook-Pro-Max exo % 
(base) satyaki_de@Satyakis-MacBook-Pro-Max exo % conda create --name exo1 python=3.13
WARNING: A conda environment already exists at '/opt/anaconda3/envs/exo1'

Remove existing environment?
This will remove ALL directories contained within this specified prefix directory, including any other conda environments.

 (y/[n])? y

Channels:
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/anaconda3/envs/exo1

  added / updated specs:
    - python=3.13


The following NEW packages will be INSTALLED:

  bzip2              pkgs/main/osx-arm64::bzip2-1.0.8-h80987f9_6 
  ca-certificates    pkgs/main/osx-arm64::ca-certificates-2025.2.25-hca03da5_0 
  expat              pkgs/main/osx-arm64::expat-2.6.4-h313beb8_0 
  libcxx             pkgs/main/osx-arm64::libcxx-14.0.6-h848a8c0_0 
  libffi             pkgs/main/osx-arm64::libffi-3.4.4-hca03da5_1 
  libmpdec           pkgs/main/osx-arm64::libmpdec-4.0.0-h80987f9_0 
  ncurses            pkgs/main/osx-arm64::ncurses-6.4-h313beb8_0 
  openssl            pkgs/main/osx-arm64::openssl-3.0.16-h02f6b3c_0 
  pip                pkgs/main/osx-arm64::pip-25.0-py313hca03da5_0 
  python             pkgs/main/osx-arm64::python-3.13.2-h4862095_100_cp313 
  python_abi         pkgs/main/osx-arm64::python_abi-3.13-0_cp313 
  readline           pkgs/main/osx-arm64::readline-8.2-h1a28f6b_0 
  setuptools         pkgs/main/osx-arm64::setuptools-75.8.0-py313hca03da5_0 
  sqlite             pkgs/main/osx-arm64::sqlite-3.45.3-h80987f9_0 
  tk                 pkgs/main/osx-arm64::tk-8.6.14-h6ba3021_0 
  tzdata             pkgs/main/noarch::tzdata-2025a-h04d1e81_0 
  wheel              pkgs/main/osx-arm64::wheel-0.45.1-py313hca03da5_0 
  xz                 pkgs/main/osx-arm64::xz-5.6.4-h80987f9_1 
  zlib               pkgs/main/osx-arm64::zlib-1.2.13-h18a0788_1 


Proceed ([y]/n)? y


Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate exo1
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) satyaki_de@Satyakis-MacBook-Pro-Max exo % conda activate exo1
(exo1) satyaki_de@Satyakis-MacBook-Pro-Max exo % 
(exo1) satyaki_de@Satyakis-MacBook-Pro-Max exo % ls -lrt
total 24576
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 docs
-rwx------  1 satyaki_de  staff     1337 Mar  9 17:06 configure_mlx.sh
-rwx------  1 satyaki_de  staff    11107 Mar  9 17:06 README.md
-rwx------  1 satyaki_de  staff    35150 Mar  9 17:06 LICENSE
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 examples
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 exo
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 extra
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 scripts
-rwx------  1 satyaki_de  staff      390 Mar  9 17:06 install.sh
-rwx------  1 satyaki_de  staff      792 Mar  9 17:06 format.py
drwx------  1 satyaki_de  staff  1048576 Mar  9 17:06 test
-rwx------  1 satyaki_de  staff     2476 Mar  9 17:06 setup.py
(exo1) satyaki_de@Satyakis-MacBook-Pro-Max exo % 
(exo1) satyaki_de@Satyakis-MacBook-Pro-Max exo % 
(exo1) satyaki_de@Satyakis-MacBook-Pro-Max exo % pip install .
Processing /Volumes/WD_BLACK/PythonCourse/Pandas/exo
  Preparing metadata (setup.py) ... done
Collecting tinygrad@ git+https://github.com/tinygrad/tinygrad.git@ec120ce6b9ce8e4ff4b5692566a683ef240e8bc8 (from exo==0.0.1)
  Cloning https://github.com/tinygrad/tinygrad.git (to revision ec120ce6b9ce8e4ff4b5692566a683ef240e8bc8) to /private/var/folders/26/dj11b57559b8r8rl6ztdpc840000gn/T/pip-install-q18fzk3r/tinygrad_7917114c483a4d9c83c795b69dbeb5c7
  Running command git clone --filter=blob:none --quiet https://github.com/tinygrad/tinygrad.git /private/var/folders/26/dj11b57559b8r8rl6ztdpc840000gn/T/pip-install-q18fzk3r/tinygrad_7917114c483a4d9c83c795b69dbeb5c7
  Running command git rev-parse -q --verify 'sha^ec120ce6b9ce8e4ff4b5692566a683ef240e8bc8'
  Running command git fetch -q https://github.com/tinygrad/tinygrad.git ec120ce6b9ce8e4ff4b5692566a683ef240e8bc8
  Running command git checkout -q ec120ce6b9ce8e4ff4b5692566a683ef240e8bc8
  Resolved https://github.com/tinygrad/tinygrad.git to commit ec120ce6b9ce8e4ff4b5692566a683ef240e8bc8
  Preparing metadata (setup.py) ... done
Collecting aiohttp==3.10.11 (from exo==0.0.1)
.
.
(Installed many more dependant packages)
.
.
Downloading propcache-0.3.0-cp313-cp313-macosx_11_0_arm64.whl (44 kB)
Building wheels for collected packages: exo, nuitka, numpy, uuid, tinygrad
  Building wheel for exo (setup.py) ... done
  Created wheel for exo: filename=exo-0.0.1-py3-none-any.whl size=901357 sha256=5665297f8ea09d06670c9dea91e40270acc4a3cf99a560bf8d268abb236050f7
  Stored in directory: /private/var/folders/26/dj118r8rl6ztdpc840000gn/T/pip-ephem-wheel-cache-0k8zloo3/wheels/b6/91/fb/c1c7d8ca90cf16b9cd8203c11bb512614bee7f6d34
  Building wheel for nuitka (pyproject.toml) ... done
  Created wheel for nuitka: filename=nuitka-2.5.1-cp313-cp313-macosx_11_0_arm64.whl size=3432720 sha256=ae5a280a1684fde98c334516ee8a99f9f0acb6fc2f625643b7f9c5c0887c2998
  Stored in directory: /Users/satyaki_de/Library/Caches/pip/wheels/f6/c9/53/9e37c6fb34c27e892e8357aaead46da610f82117ab2825
  Building wheel for numpy (pyproject.toml) ... done
  Created wheel for numpy: filename=numpy-2.0.0-cp313-cp313-macosx_15_0_arm64.whl size=4920701 sha256=f030b0aa51ec6628f708fab0af14ff765a46d210df89aa66dd8d9482e59b5
  Stored in directory: /Users/satyaki_de/Library/Caches/pip/wheels/e0/d3/66/30d07c18e56ac85e8d3ceaf22f093a09bae124a472b85d1
  Building wheel for uuid (setup.py) ... done
  Created wheel for uuid: filename=uuid-1.30-py3-none-any.whl size=6504 sha256=885103a90d1dc92d9a75707fc353f4154597d232f2599a636de1bc6d1c83d
  Stored in directory: /Users/satyaki_de/Library/Caches/pip/wheels/cc/9d/72/13ff6a181eacfdbd6d761a4ee7c5c9f92034a9dc8a1b3c
  Building wheel for tinygrad (setup.py) ... done
  Created wheel for tinygrad: filename=tinygrad-0.10.0-py3-none-any.whl size=1333964 sha256=1f08c5ce55aa3c87668675beb80810d609955a81b99d416459d2489b36a
  Stored in directory: /Users/satyaki_de/Library/Caches/pip/wheels/c7/bd/02/bd91c1303002619dad23f70f4c1f1c15d0c24c60b043e
Successfully built exo nuitka numpy uuid tinygrad
Installing collected packages: uuid, sentencepiece, nvidia-ml-py, zstandard, uvloop, urllib3, typing-extensions, tqdm, tinygrad, scapy, safetensors, regex, pyyaml, pygments, psutil, protobuf, propcache, prometheus-client, pillow, packaging, ordered-set, numpy, multidict, mlx, mdurl, MarkupSafe, idna, grpcio, fsspec, frozenlist, filelock, charset-normalizer, certifi, attrs, annotated-types, aiohappyeyeballs, aiofiles, yarl, requests, pydantic-core, opencv-python, nuitka, markdown-it-py, Jinja2, grpcio-tools, aiosignal, rich, pydantic, huggingface-hub, aiohttp, tokenizers, aiohttp_cors, transformers, mlx-lm, exo
Successfully installed Jinja2-3.1.4 MarkupSafe-3.0.2 aiofiles-24.1.0 aiohappyeyeballs-2.5.0 aiohttp-3.10.11 aiohttp_cors-0.7.0 aiosignal-1.3.2 annotated-types-0.7.0 attrs-25.1.0 certifi-2025.1.31 charset-normalizer-3.4.1 exo-0.0.1 filelock-3.17.0 frozenlist-1.5.0 fsspec-2025.3.0 grpcio-1.67.0 grpcio-tools-1.67.0 huggingface-hub-0.29.2 idna-3.10 markdown-it-py-3.0.0 mdurl-0.1.2 mlx-0.22.0 mlx-lm-0.21.1 multidict-6.1.0 nuitka-2.5.1 numpy-2.0.0 nvidia-ml-py-12.560.30 opencv-python-4.10.0.84 ordered-set-4.1.0 packaging-24.2 pillow-10.4.0 prometheus-client-0.20.0 propcache-0.3.0 protobuf-5.28.1 psutil-6.0.0 pydantic-2.9.2 pydantic-core-2.23.4 pygments-2.19.1 pyyaml-6.0.2 regex-2024.11.6 requests-2.32.3 rich-13.7.1 safetensors-0.5.3 scapy-2.6.1 sentencepiece-0.2.0 tinygrad-0.10.0 tokenizers-0.20.3 tqdm-4.66.4 transformers-4.46.3 typing-extensions-4.12.2 urllib3-2.3.0 uuid-1.30 uvloop-0.21.0 yarl-1.18.3 zstandard-0.23.0
(exo1) satyaki_de@Satyakis-MacBook-Pro-Max exo % 

And, you need to perform the same process in other available devices as well.

Now, we’re ready to proceed with the final command –

(.venv) (exo1) satyaki_de@Satyakis-MacBook-Pro-Max exo % exo
/opt/anaconda3/envs/exo1/lib/python3.13/site-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.1 at node_service.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
  warnings.warn(
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Selected inference engine: None

  _____  _____  
 / _ \ \/ / _ \ 
|  __/>  < (_) |
 \___/_/\_\___/ 
    
Detected system: Apple Silicon Mac
Inference engine name after selection: mlx
Using inference engine: MLXDynamicShardInferenceEngine with shard downloader: SingletonShardDownloader
[60771, 54631, 54661]
Chat interface started:
 - http://127.0.0.1:52415
 - http://XXX.XXX.XX.XX:52415
 - http://XXX.XXX.XXX.XX:52415
 - http://XXX.XXX.XXX.XXX:52415
ChatGPT API endpoint served at:
 - http://127.0.0.1:52415/v1/chat/completions
 - http://XXX.XXX.X.XX:52415/v1/chat/completions
 - http://XXX.XXX.XXX.XX:52415/v1/chat/completions
 - http://XXX.XXX.XXX.XXX:52415/v1/chat/completions
has_read=True, has_write=True
╭────────────────────────────────────────────────────────────────────────────────────────────── Exo Cluster (2 nodes) ───────────────────────────────────────────────────────────────────────────────────────────────╮
Received exit signal SIGTERM...
Thank you for using exo.

  _____  _____  
 / _ \ \/ / _ \ 
|  __/>  < (_) |
 \___/_/\_\___/ 
    

Note that I’ve masked the IP addresses for security reasons.


At the beginning, if we trigger the main MacBook Pro Max, the “Exo” screen should looks like this –

And if you open the URL, you will see the following ChatGPT-like interface –

Connecting without the Thunderbolt bridge with the relevant port or a hub may cause performance degradation. Hence, how you connect will play a major role in the success of this intention. However, this is certainly a great idea to proceed with.


So, we’ve done it.

We’ll cover the detailed performance testing, Optimized configurations & many other useful details in our next post.

Till then, Happy Avenging! 🙂

Monitoring & evaluating the leading LLMs (both the established & new) by Python-based evaluator

As we’re leaping more & more into the field of Generative AI, one of the frequent questions or challenges people are getting more & more is the performance & other evaluation factors. These factors will eventually bring the fruit of this technology; otherwise, you will end up in technical debt.

This post will discuss the key snippets of the monitoring app based on the Python-based AI app. But before that, let us first view the demo.

Isn’t it exciting?


Let us deep dive into it. But, here is the flow this solution will follow.

So, the current application will invoke the industry bigshots and some relatively unknown or new LLMs.

In this case, we’ll evaluate Anthropic, Open AI, DeepSeek, and Bharat GPT’s various models. However, Bharat GPT is open source, so we’ll use the Huggingface library and execute it locally against my MacBook Pro M4 Max.

The following are the KPIs we’re going to evaluate:

Here are the lists of dependant python packages that is require to run this application –

pip install certifi==2024.8.30
pip install anthropic==0.42.0
pip install huggingface-hub==0.27.0
pip install nltk==3.9.1
pip install numpy==2.2.1
pip install moviepy==2.1.1
pip install numpy==2.1.3
pip install openai==1.59.3
pip install pandas==2.2.3
pip install pillow==11.1.0
pip install pip==24.3.1
pip install psutil==6.1.1
pip install requests==2.32.3
pip install rouge_score==0.1.2
pip install scikit-learn==1.6.0
pip install setuptools==70.2.0
pip install tokenizers==0.21.0
pip install torch==2.6.0.dev20250104
pip install torchaudio==2.6.0.dev20250104
pip install torchvision==0.22.0.dev20250104
pip install tqdm==4.67.1
pip install transformers==4.47.1
    @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
    def get_claude_response(self, prompt: str) -> str:
        response = self.anthropic_client.messages.create(
            model=anthropic_model,
            max_tokens=maxToken,
            messages=[{"role": "user", "content": prompt}]
        )
        return response.content[0].text
  1. The Retry Mechanism
    • The @retry line means this function will automatically try again if it fails.
    • It will stop retrying after 3 attempts (stop_after_attempt(3)).
    • It will wait longer between retries, starting at 4 seconds and increasing up to 10 seconds (wait_exponential(multiplier=1, min=4, max=10)).
  2. The Function Purpose
    • The function takes a message, called prompt, as input (a string of text).
    • It uses a service (likely an AI system like Claude) to generate a response to this prompt.
  3. Sending the Message
    • Inside the function, the code self.anthropic_client.messages.create is the part that actually sends the prompt to the AI.
    • It specifies:Which AI model to use (e.g., anthropic_model).
    • The maximum length of the response (controlled by maxToken).
    • The input message for the AI has a “role” (user), as well as the content of the prompt.
  4. Getting the Response
    • Once the AI generates a response, it’s saved as response.
    • The code retrieves the first part of the response (response.content[0].text) and sends it back to whoever called the function.

Similarly, it will work for Open AI as well.

    @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
    def get_deepseek_response(self, prompt: str) -> tuple:
        deepseek_api_key = self.deepseek_api_key

        headers = {
            "Authorization": f"Bearer {deepseek_api_key}",
            "Content-Type": "application/json"
            }
        
        payload = {
            "model": deepseek_model,  
            "messages": [{"role": "user", "content": prompt}],
            "max_tokens": maxToken
            }
        
        response = requests.post(DEEPSEEK_API_URL, headers=headers, json=payload)

        if response.status_code == 200:
            res = response.json()["choices"][0]["message"]["content"]
        else:
            res = "API request failed with status code " + str(response.status_code) + ":" + str(response.text)

        return res
  1. Retry Mechanism:
    • The @retry line ensures the function will try again if it fails.
    • It will stop retrying after 3 attempts (stop_after_attempt(3)).
    • It waits between retries, starting at 4 seconds and increasing up to 10 seconds (wait_exponential(multiplier=1, min=4, max=10)).

  1. What the Function Does:
    • The function takes one input, prompt, which is the message or question you want to send to the AI.
    • It returns the AI’s response or an error message.

  1. Preparing to Communicate with the API:
    • API Key: It gets the API key for the DeepSeek service from self.deepseek_api_key.
    • Headers: These tell the API that the request will use the API key (for security) and that the data format is JSON (structured text).
    • Payload: This is the information sent to the AI. It includes:
      • Model: Specifies which version of the AI to use (deepseek_model).
      • Messages: The input message with the role “user” and your prompt.
      • Max Tokens: Defines the maximum size of the AI’s response (maxToken).

  1. Sending the Request:
    • It uses the requests.post() method to send the payload and headers to the DeepSeek API using the URL DEEPSEEK_API_URL.

  1. Processing the Response:
    • If the API responds successfully (status_code == 200):
      • It extracts the AI’s reply from the response data.
      • Specifically, it gets the first choice’s message content: response.json()["choices"][0]["message"]["content"].
    • If there’s an error:
      • It constructs an error message with the status code and detailed error text from the API.

  1. Returning the Result:
    • The function outputs either the AI’s response or the error message.
    @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
    def get_bharatgpt_response(self, prompt: str) -> tuple:
        try:
            messages = [[{"role": "user", "content": prompt}]]
            
            response = pipe(messages, max_new_tokens=maxToken,)

            # Extract 'content' field safely
            res = next((entry.get("content", "")
                        for entry in response[0][0].get("generated_text", [])
                        if isinstance(entry, dict) and entry.get("role") == "assistant"
                        ),
                        None,
                        )
            
            return res
        except Exception as e:
            x = str(e)
            print('Error: ', x)

            return ""
  1. Retry Mechanism:The @retry ensures the function will try again if it fails.
    • It will stop retrying after 3 attempts (stop_after_attempt(3)).
    • The waiting time between retries starts at 4 seconds and increases exponentially up to 10 seconds (wait_exponential(multiplier=1, min=4, max=10)).
  2. What the Function Does:The function takes one input, prompt, which is the message or question you want to send to BharatGPT.
    • It returns the AI’s response or an empty string if something goes wrong.
  3. Sending the Prompt:Messages Structure: The function wraps the user’s prompt in a format that the BharatGPT AI understands:
    • messages = [[{"role": "user", "content": prompt}]]
    • This tells the AI that the prompt is coming from the “user.”
  4. Pipe Function: It uses a pipe() method to send the messages to the AI system.
    • max_new_tokens=maxToken: Limits how long the AI’s response can be.
  5. Extracting the Response:The response from the AI is in a structured format. The code looks for the first piece of text where:
    • The role is “assistant” (meaning it’s the AI’s reply).
    • The text is in the “content” field.
    • The next() function safely extracts this “content” field or returns None if it can’t find it.
  6. Error Handling:If something goes wrong (e.g., the AI doesn’t respond or there’s a technical issue), the code:
    • Captures the error message in e.
    • Prints the error message: print('Error: ', x).
    • Returns an empty string ("") instead of crashing.
  7. Returning the Result:If everything works, the function gives you the AI’s response as plain text.
    • If there’s an error, it gives you an empty string, indicating no response was received.

    def get_model_response(self, model_name: str, prompt: str) -> ModelResponse:
        """Get response from specified model with metrics"""
        start_time = time.time()
        start_memory = psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024

        try:
            if model_name == "claude-3":
                response_content = self.get_claude_response(prompt)
            elif model_name == "gpt4":
                response_content = self.get_gpt4_response(prompt)
            elif model_name == "deepseek-chat":
                response_content = self.get_deepseek_response(prompt)
            elif model_name == "bharat-gpt":
                response_content = self.get_bharatgpt_response(prompt)

            # Model-specific API calls 
            token_count = len(self.bert_tokenizer.encode(response_content))
            
            end_memory = psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024
            memory_usage = end_memory - start_memory
            
            return ModelResponse(
                content=response_content,
                response_time=time.time() - start_time,
                token_count=token_count,
                memory_usage=memory_usage
            )
        except Exception as e:
            logging.error(f"Error getting response from {model_name}: {str(e)}")
            return ModelResponse(
                content="",
                response_time=0,
                token_count=0,
                memory_usage=0,
                error=str(e)
            )

Start Tracking Time and Memory:

    • The function starts a timer (start_time) to measure how long it takes to get a response.
    • It also checks how much memory is being used at the beginning (start_memory).

    Choose the AI Model:

    • Based on the model_name provided, the function selects the appropriate method to get a response:
      • "claude-3" → Calls get_claude_response(prompt).
      • "gpt4" → Calls get_gpt4_response(prompt).
      • "deepseek-chat" → Calls get_deepseek_response(prompt).
      • "bharat-gpt" → Calls get_bharatgpt_response(prompt).

    Process the Response:

    • Once the response is received, the function calculates:
      • Token Count: The number of tokens (small chunks of text) in the response using a tokenizer.
      • Memory Usage: The difference between memory usage after the response (end_memory) and before it (start_memory).

    Return the Results:

    • The function bundles all the information into a ModelResponse object:
      • The AI’s reply (content).
      • How long the response took (response_time).
      • The number of tokens in the reply (token_count).
      • How much memory was used (memory_usage).

    Handle Errors:

    • If something goes wrong (e.g., the AI doesn’t respond), the function:
      • Logs the error message.
      • Returns an empty response with default values and the error message.
        def evaluate_text_quality(self, generated: str, reference: str) -> Dict[str, float]:
            """Evaluate text quality metrics"""
            # BERTScore
            gen_embedding = self.sentence_model.encode([generated])
            ref_embedding = self.sentence_model.encode([reference])
            bert_score = cosine_similarity(gen_embedding, ref_embedding)[0][0]
    
            # BLEU Score
            generated_tokens = word_tokenize(generated.lower())
            reference_tokens = word_tokenize(reference.lower())
            bleu = sentence_bleu([reference_tokens], generated_tokens)
    
            # METEOR Score
            meteor = meteor_score([reference_tokens], generated_tokens)
    
            return {
                'bert_score': bert_score,
                'bleu_score': bleu,
                'meteor_score': meteor
            }

    Inputs:

    • generated: The text produced by the AI.
    • reference: The correct or expected version of the text.

    Calculating BERTScore:

    • Converts the generated and reference texts into numerical embeddings (mathematical representations) using a pre-trained model (self.sentence_model.encode).
    • Measures the similarity between the two embeddings using cosine similarity. This gives the bert_score, which ranges from -1 (completely different) to 1 (very similar).

    Calculating BLEU Score:

    • Breaks the generated and reference texts into individual words (tokens) using word_tokenize.
    • Converts both texts to lowercase for consistent comparison.
    • Calculates the BLEU Score (sentence_bleu), which checks how many words or phrases in the generated text overlap with the reference. BLEU values range from 0 (no match) to 1 (perfect match).

    Calculating METEOR Score:

    • Also uses the tokenized versions of generated and reference texts.
    • Calculates the METEOR Score (meteor_score), which considers exact matches, synonyms, and word order. Scores range from 0 (no match) to 1 (perfect match).

    Returning the Results:

    • Combines the three scores into a dictionary with the keys 'bert_score', 'bleu_score', and 'meteor_score'.

    Similarly, other functions are developed.

        def run_comprehensive_evaluation(self, evaluation_data: List[Dict]) -> pd.DataFrame:
            """Run comprehensive evaluation on all metrics"""
            results = []
            
            for item in evaluation_data:
                prompt = item['prompt']
                reference = item['reference']
                task_criteria = item.get('task_criteria', {})
                
                for model_name in self.model_configs.keys():
                    # Get multiple responses to evaluate reliability
                    responses = [
                        self.get_model_response(model_name, prompt)
                        for _ in range(3)  # Get 3 responses for reliability testing
                    ]
                    
                    # Use the best response for other evaluations
                    best_response = max(responses, key=lambda x: len(x.content) if not x.error else 0)
                    
                    if best_response.error:
                        logging.error(f"Error in model {model_name}: {best_response.error}")
                        continue
                    
                    # Gather all metrics
                    metrics = {
                        'model': model_name,
                        'prompt': prompt,
                        'response': best_response.content,
                        **self.evaluate_text_quality(best_response.content, reference),
                        **self.evaluate_factual_accuracy(best_response.content, reference),
                        **self.evaluate_task_performance(best_response.content, task_criteria),
                        **self.evaluate_technical_performance(best_response),
                        **self.evaluate_reliability(responses),
                        **self.evaluate_safety(best_response.content)
                    }
                    
                    # Add business impact metrics using task performance
                    metrics.update(self.evaluate_business_impact(
                        best_response,
                        metrics['task_completion']
                    ))
                    
                    results.append(metrics)
            
            return pd.DataFrame(results)
    • Input:
      • evaluation_data: A list of test cases, where each case is a dictionary containing:
        • prompt: The question or input to the AI model.
        • reference: The ideal or expected answer.
        • task_criteria (optional): Additional rules or requirements for the task.
    • Initialize Results:
      • An empty list results is created to store the evaluation metrics for each model and test case.
    • Iterate Through Test Cases:
      • For each item in the evaluation_data:
        • Extract the prompt, reference, and task_criteria.
    • Evaluate Each Model:
      • Loop through all available AI models (self.model_configs.keys()).
      • Generate three responses for each model to test reliability.
    • Select the Best Response:
      • Out of the three responses, pick the one with the most content (best_response), ignoring responses with errors.
    • Handle Errors:
      • If a response has an error, log the issue and skip further evaluation for that model.
    • Evaluate Metrics:
      • Using the best_response, calculate a variety of metrics, including:
        • Text Quality: How similar the response is to the reference.
        • Factual Accuracy: Whether the response is factually correct.
        • Task Performance: How well it meets task-specific criteria.
        • Technical Performance: Evaluate time, memory, or other system-related metrics.
        • Reliability: Check consistency across multiple responses.
        • Safety: Ensure the response is safe and appropriate.
    • Evaluate Business Impact:
      • Add metrics for business impact (e.g., how well the task was completed, using task_completion as a key factor).
    • Store Results:
      • Add the calculated metrics for this model and prompt to the results list.
    • Return Results as a DataFrame:
      • Convert the results list into a structured table (a pandas DataFrame) for easy analysis and visualization.

    Great! So, now, we’ve explained the code.

    Let us understand the final outcome of this run & what we can conclude from that.

    1. BERT Score (Semantic Understanding):
      • GPT4 leads slightly at 0.8322 (83.22%)
      • Bharat-GPT close second at 0.8118 (81.18%)
      • Claude-3 at 0.8019 (80.19%)
      • DeepSeek-Chat at 0.7819 (78.19%) Think of this like a “comprehension score” – how well the models understand the context. All models show strong understanding, with only a 5% difference between best and worst.
    2. BLEU Score (Word-for-Word Accuracy):
      • Bharat-GPT leads at 0.0567 (5.67%)
      • Claude-3 at 0.0344 (3.44%)
      • GPT4 at 0.0306 (3.06%)
      • DeepSeek-Chat lowest at 0.0189 (1.89%) These low scores suggest models use different wording than references, which isn’t necessarily bad.
    3. METEOR Score (Meaning Preservation):
      • Bharat-GPT leads at 0.4684 (46.84%)
      • Claude-3 close second at 0.4507 (45.07%)
      • GPT4 at 0.2960 (29.60%)
      • DeepSeek-Chat at 0.2652 (26.52%) This shows how well models maintain meaning while using different words.
    4. Response Time (Speed):
      • Claude-3 fastest: 4.40 seconds
      • Bharat-GPT: 6.35 seconds
      • GPT4: 6.43 seconds
      • DeepSeek-Chat slowest: 8.52 seconds
    5. Safety and Reliability:
      • Error Rate: Perfect 0.0 for all models
      • Toxicity: All very safe (below 0.15%) 
        • Claude-3 safest at 0.0007GPT4 at 0.0008Bharat-GPT at 0.0012
        • DeepSeek-Chat at 0.0014
    6. Cost Efficiency:
      • Claude-3 most economical: $0.0019 per response
      • Bharat-GPT close: $0.0021
      • GPT4: $0.0038
      • DeepSeek-Chat highest: $0.0050

    Key Takeaways by Model:

    1. Claude-3: ✓ Fastest responses ✓ Most cost-effective ✓ Excellent meaning preservation ✓ Lowest toxicity
    2. Bharat-GPT: ✓ Best BLEU and METEOR scores ✓ Strong semantic understanding ✓ Cost-effective ✗ Moderate response time
    3. GPT4: ✓ Best semantic understanding ✓ Good safety metrics ✗ Higher cost ✗ Moderate response time
    4. DeepSeek-Chat: ✗ Generally lower performance ✗ Slowest responses ✗ Highest cost ✗ Slightly higher toxicity

    Reliability of These Statistics:

    Strong Points:

    • Comprehensive metric coverage
    • Consistent patterns across evaluations
    • Zero error rates show reliability
    • Clear differentiation between models

    Limitations:

    • BLEU scores are quite low across all models
    • Doesn’t measure creative or innovative responses
    • May not reflect specific use case performance
    • Single snapshot rather than long-term performance

    Final Observation:

    1. Best Overall Value: Claude-3
      • Fast, cost-effective, safe, good performance
    2. Best for Accuracy: Bharat-GPT
      • Highest meaning preservation and precision
    3. Best for Understanding: GPT4
      • Strongest semantic comprehension
    4. Consider Your Priorities: 
      • Speed → Choose Claude-3
      • Cost → Choose Claude-3 or Bharat-GPT
      • Accuracy → Choose Bharat-GPT
      • Understanding → Choose GPT4

    These statistics provide reliable comparative data but should be part of a broader decision-making process that includes your specific needs, budget, and use cases.


    For the Bharat GPT model, we’ve tested this locally on my MacBook Pro 4 Max. And, the configuration is as follows –

    I’ve tried the API version locally, & it provided a similar performance against the stats that we received by running locally. Unfortunately, they haven’t made the API version public yet.

    So, apart from the Anthropic & Open AI, I’ll watch this new LLM (Bharat GPT) for overall stats in the coming days.


    So, we’ve done it.

    You can find the detailed code at the GitHub link.

    I’ll bring some more exciting topics in the coming days from the Python verse.

    Till then, Happy Avenging! 🙂

    Enabling & Exploring Stable Defussion – Part 3

    Before we dive into the details of this post, let us provide the previous two links that precede it.

    Enabling & Exploring Stable Defussion – Part 1

    Enabling & Exploring Stable Defussion – Part 2

    For, reference, we’ll share the demo before deep dive into the actual follow-up analysis in the below section –


    Now, let us continue our discussions from where we left.

    class clsText2Image:
        def __init__(self, pipe, output_path, filename):
    
            self.pipe = pipe
            
            # More aggressive attention slicing
            self.pipe.enable_attention_slicing(slice_size=1)
    
            self.output_path = f"{output_path}{filename}"
            
            # Warm up the pipeline
            self._warmup()
        
        def _warmup(self):
            """Warm up the pipeline to optimize memory allocation"""
            with torch.no_grad():
                _ = self.pipe("warmup", num_inference_steps=1, height=512, width=512)
            torch.mps.empty_cache()
            gc.collect()
        
        def generate(self, prompt, num_inference_steps=12, guidance_scale=3.0):
            try:
                torch.mps.empty_cache()
                gc.collect()
                
                with torch.autocast(device_type="mps"):
                    with torch.no_grad():
                        image = self.pipe(
                            prompt,
                            num_inference_steps=num_inference_steps,
                            guidance_scale=guidance_scale,
                            height=1024,
                            width=1024,
                        ).images[0]
                
                image.save(self.output_path)
                return 0
            except Exception as e:
                print(f'Error: {str(e)}')
                return 1
            finally:
                torch.mps.empty_cache()
                gc.collect()
    
        def genImage(self, prompt):
            try:
    
                # Initialize generator
                x = self.generate(prompt)
    
                if x == 0:
                    print('Successfully processed first pass!')
                else:
                    print('Failed complete first pass!')
                    raise 
    
                return 0
    
            except Exception as e:
                print(f"\nAn unexpected error occurred: {str(e)}")
    
                return 1

    This is the initialization method for the clsText2Image class:

    • Takes a pre-configured pipe (text-to-image pipeline), an output_path, and a filename.
    • Enables more aggressive memory optimization by setting “attention slicing.”
    • Prepares the full file path for saving generated images.
    • Calls a _warmup method to pre-load the pipeline and optimize memory allocation.

    This private method warms up the pipeline:

    • Sends a dummy “warmup” request with basic parameters to allocate memory efficiently.
    • Clears any cached memory (torch.mps.empty_cache()) and performs garbage collection (gc.collect()).
    • Ensures smoother operation for future image generation tasks.

    This method generates an image from a text prompt:

    • Clears memory cache and performs garbage collection before starting.
    • Uses the text-to-image pipeline (pipe) to generate an image:
      • Takes the prompt, number of inference steps, and guidance scale as input.
      • Outputs an image at 1024×1024 resolution.
    • Saves the generated image to the specified output path.
    • Returns 0 on success or 1 on failure.
    • Ensures cleanup by clearing memory and collecting garbage, even in case of errors.

    This method simplifies image generation:

    • Calls the generate method with the given prompt.
    • Prints a success message if the image is generated (0 return value).
    • On failure, logs the error and raises an exception.
    • Returns 0 on success or 1 on failure.
    class clsImage2Video:
        def __init__(self, pipeline):
            
            # Optimize model loading
            torch.mps.empty_cache()
            self.pipeline = pipeline
    
        def generate_frames(self, pipeline, init_image, prompt, duration_seconds=10):
            try:
                torch.mps.empty_cache()
                gc.collect()
    
                base_frames = []
                img = Image.open(init_image).convert("RGB").resize((1024, 1024))
                
                for _ in range(10):
                    result = pipeline(
                        prompt=prompt,
                        image=img,
                        strength=0.45,
                        guidance_scale=7.5,
                        num_inference_steps=25
                    ).images[0]
    
                    base_frames.append(np.array(result))
                    img = result
                    torch.mps.empty_cache()
    
                frames = []
                for i in range(len(base_frames)-1):
                    frame1, frame2 = base_frames[i], base_frames[i+1]
                    for t in np.linspace(0, 1, int(duration_seconds*24/10)):
                        frame = (1-t)*frame1 + t*frame2
                        frames.append(frame.astype(np.uint8))
                
                return frames
            except Exception as e:
                frames = []
                print(f'Error: {str(e)}')
    
                return frames
            finally:
                torch.mps.empty_cache()
                gc.collect()
    
        # Main method
        def genVideo(self, prompt, inputImage, targetVideo, fps):
            try:
                print("Starting animation generation...")
                
                init_image_path = inputImage
                output_path = targetVideo
                fps = fps
                
                frames = self.generate_frames(
                    pipeline=self.pipeline,
                    init_image=init_image_path,
                    prompt=prompt,
                    duration_seconds=20
                )
                
                imageio.mimsave(output_path, frames, fps=30)
    
                print("Animation completed successfully!")
    
                return 0
            except Exception as e:
                x = str(e)
                print('Error: ', x)
    
                return 1

    This initializes the clsImage2Video class:

    • Clears the GPU cache to optimize memory before loading.
    • Sets up the pipeline for generating frames, which uses an image-to-video transformation model.

    This function generates frames for a video:

    • Starts by clearing GPU memory and running garbage collection.
    • Loads the init_image, resizes it to 1024×1024 pixels, and converts it to RGB format.
    • Iteratively applies the pipeline to transform the image:
      • Uses the prompt and specified parameters like strength, guidance_scale, and num_inference_steps.
      • Stores the resulting frames in a list.
    • Interpolates between consecutive frames to create smooth transitions:
      • Uses linear blending for smooth animation across a specified duration and frame rate (24 fps for 10 segments).
    • Returns the final list of generated frames or an empty list if an error occurs.
    • Always clears memory after execution.

    This is the main function for creating a video from an image and text prompt:

    • Logs the start of the animation generation process.
    • Calls generate_frames() with the given pipeline, inputImage, and prompt to create frames.
    • Saves the generated frames as a video using the imageio library, setting the specified frame rate (fps).
    • Logs a success message and returns 0 if the process is successful.
    • On error, logs the issue and returns 1.

    Now, let us understand the performance. But, before that let us explore the device on which we’ve performed these stress test that involves GPU & CPUs as well.

    And, here is the performance stats –

    From the above snapshot, we can clearly communicate that the GPU is 100% utilized. However, the CPU has shown a significant % of availability.

    As you can see, the first pass converts the input prompt to intermediate images within 1 min 30 sec. However, the second pass constitutes multiple hops (11 hops) on an avg 22 seconds. Overall, the application will finish in 5 minutes 36 seconds for a 10-second video clip.


    So, we’ve done it.

    You can find the detailed code at the GitHub link.

    I’ll bring some more exciting topics in the coming days from the Python verse.

    Till then, Happy Avenging! 🙂

    Enabling & Exploring Stable Defussion – Part 2

    As we’ve started explaining, the importance & usage of Stable Defussion in our previous post:

    Enabling & Exploring Stable Defussion – Part 1

    In today’s post, we’ll discuss another approach, where we built the custom Python-based SDK solution that consumes HuggingFace Library, which generates video out of the supplied prompt.

    But, before that, let us view the demo generated from a custom solution.

    Isn’t it exciting? Let us dive deep into the details.


    Let us understand basic flow of events for the custom solution –

    So, the application will interact with the python-sdk like “stable-diffusion-3.5-large” & “dreamshaper-xl-1-0”, which is available in HuggingFace. As part of the process, these libraries will load all the large models inside the local laptop that require some time depend upon the bandwidth of your internet.

    Before we even deep dive into the code, let us understand the flow of Python scripts as shown below:

    From the above diagram, we can understand that the main application will be triggered by “generateText2Video.py”. As you can see that “clsConfigClient.py” has all the necessary parameter information that will be supplied to all the scripts.

    “generateText2Video.py” will trigger the main class named “clsText2Video.py”, which then calls all the subsequent classes.

    Great! Since we now have better visibility of the script flow, let’s examine the key snippets individually.


    class clsText2Video:
        def __init__(self, model_id_1, model_id_2, output_path, filename, vidfilename, fps, force_cpu=False):
            self.model_id_1 = model_id_1
            self.model_id_2 = model_id_2
            self.output_path = output_path
            self.filename = filename
            self.vidfilename = vidfilename
            self.force_cpu = force_cpu
            self.fps = fps
    
            # Initialize in main process
            os.environ["TOKENIZERS_PARALLELISM"] = "true"
            self.r1 = cm.clsMaster(force_cpu)
            self.torch_type = self.r1.getTorchType()
            
            torch.mps.empty_cache()
            self.pipe = self.r1.getText2ImagePipe(self.model_id_1, self.torch_type)
            self.pipeline = self.r1.getImage2VideoPipe(self.model_id_2, self.torch_type)
    
            self.text2img = cti.clsText2Image(self.pipe, self.output_path, self.filename)
            self.img2vid = civ.clsImage2Video(self.pipeline)
    
        def getPrompt2Video(self, prompt):
            try:
                input_image = self.output_path + self.filename
                target_video = self.output_path + self.vidfilename
    
                if self.text2img.genImage(prompt) == 0:
                    print('Pass 1: Text to intermediate images generated!')
                    
                    if self.img2vid.genVideo(prompt, input_image, target_video, self.fps) == 0:
                        print('Pass 2: Successfully generated!')
                        return 0
                return 1
            except Exception as e:
                print(f"\nAn unexpected error occurred: {str(e)}")
                return 1

    Now, let us interpret:

    This is the initialization method for the class. It does the following:

    • Sets up configurations like model IDs, output paths, filenames, video filename, frames per second (fps), and whether to use the CPU (force_cpu).
    • Configures an environment variable for tokenizer parallelism.
    • Initializes helper classes (clsMaster) to manage system resources and retrieve appropriate PyTorch settings.
    • Creates two pipelines:
      • pipe: For converting text to images using the first model.
      • pipeline: For converting images to video using the second model.
    • Initializes text2img and img2vid objects:
      • text2img handles text-to-image conversions.
      • img2vid handles image-to-video conversions.

    This method generates a video from a text prompt in two steps:

    1. Text-to-Image Conversion:
      • Calls genImage(prompt) using the text2img object to create an intermediate image file.
      • If successful, it prints confirmation.
    2. Image-to-Video Conversion:
      • Uses the img2vid object to convert the intermediate image into a video file.
      • Includes the input image path, target video path, and frames per second (fps).
      • If successful, it prints confirmation.
    • If either step fails, the method returns 1.
    • Logs any unexpected errors and returns 1 in such cases.
    # Set device for Apple Silicon GPU
    def setup_gpu(force_cpu=False):
        if not force_cpu and torch.backends.mps.is_available() and torch.backends.mps.is_built():
            print('Running on Apple Silicon MPS GPU!')
            return torch.device("mps")
        return torch.device("cpu")
    
    ######################################
    ####         Global Flag      ########
    ######################################
    
    class clsMaster:
        def __init__(self, force_cpu=False):
            self.device = setup_gpu(force_cpu)
    
        def getTorchType(self):
            try:
                # Check if MPS (Apple Silicon GPU) is available
                if not torch.backends.mps.is_available():
                    torch_dtype = torch.float32
                    raise RuntimeError("MPS (Metal Performance Shaders) is not available on this system.")
                else:
                    torch_dtype = torch.float16
                
                return torch_dtype
            except Exception as e:
                torch_dtype = torch.float16
                print(f'Error: {str(e)}')
    
                return torch_dtype
    
        def getText2ImagePipe(self, model_id, torchType):
            try:
                device = self.device
    
                torch.mps.empty_cache()
                self.pipe = StableDiffusion3Pipeline.from_pretrained(model_id, torch_dtype=torchType, use_safetensors=True, variant="fp16",).to(device)
    
                return self.pipe
            except Exception as e:
                x = str(e)
                print('Error: ', x)
    
                torch.mps.empty_cache()
                self.pipe = StableDiffusion3Pipeline.from_pretrained(model_id, torch_dtype=torchType,).to(device)
    
                return self.pipe
            
        def getImage2VideoPipe(self, model_id, torchType):
            try:
                device = self.device
    
                torch.mps.empty_cache()
                self.pipeline = StableDiffusionXLImg2ImgPipeline.from_pretrained(model_id, torch_dtype=torchType, use_safetensors=True, use_fast=True).to(device)
    
                return self.pipeline
            except Exception as e:
                x = str(e)
                print('Error: ', x)
    
                torch.mps.empty_cache()
                self.pipeline = StableDiffusionXLImg2ImgPipeline.from_pretrained(model_id, torch_dtype=torchType).to(device)
    
                return self.pipeline

    Let us interpret:

    This function determines whether to use the Apple Silicon GPU (MPS) or the CPU:

    • If force_cpu is False and the MPS GPU is available, it sets the device to “mps” (Apple GPU) and prints a message.
    • Otherwise, it defaults to the CPU.

    This is the initializer for the clsMaster class:

    • It sets the device to either GPU or CPU using the setup_gpu function (mentioned above) based on the force_cpu flag.

    This method determines the PyTorch data type to use:

    • Checks if MPS GPU is available:
      • If available, uses torch.float16 for optimized performance.
      • If unavailable, defaults to torch.float32 and raises a warning.
    • Handles errors gracefully by defaulting to torch.float16 and printing the error.

    This method initializes a text-to-image pipeline:

    • Loads the Stable Diffusion model with the given model_id and torchType.
    • Configures it for MPS GPU or CPU, based on the device.
    • Clears the GPU cache before loading the model to optimize memory usage.
    • If an error occurs, attempts to reload the pipeline without safetensors.

    This method initializes an image-to-video pipeline:

    • Similar to getText2ImagePipe, it loads the Stable Diffusion XL Img2Img pipeline with the specified model_id and torchType.
    • Configures it for MPS GPU or CPU and clears the cache before loading.
    • On error, reloads the pipeline without additional optimization settings and prints the error.

    Let us continue this in the next post:

    Enabling & Exploring Stable Defussion – Part 3

    Till then, Happy Avenging! 🙂

    Building solutions using LLM AutoGen in Python – Part 3

    Before we dive into the details of this post, let us provide the previous two links that precede it.

    Building solutions using LLM AutoGen in Python – Part 1

    Building solutions using LLM AutoGen in Python – Part 2

    For, reference, we’ll share the demo before deep dive into the actual follow-up analysis in the below section –


    In this post, we will understand the initial code generated & then the revised code to compare them for a better understanding of the impact of revised prompts.

    But, before that let us broadly understand the communication types between the agents.

    • Agents Involved: Agent1, Agent2
    • Flow:
      • Agent1 sends a request directly to Agent2.
      • Agent2 processes the request and sends the response back to Agent1.
    • Use Case: Simple query-response interactions without intermediaries.
    • Agents Involved: UserAgent, Mediator, SpecialistAgent1, SpecialistAgent2
    • Flow:
      • UserAgent sends input to Mediator.
      • Mediator delegates tasks to SpecialistAgent1 and SpecialistAgent2.
      • Specialists process tasks and return results to Mediator.
      • Mediator consolidates results and sends them back to UserAgent.
    • Agents Involved: Broadcaster, AgentA, AgentB, AgentC
    • Flow:
      • Broadcaster sends a message to multiple agents simultaneously.
      • Agents that find the message relevant (AgentA, AgentC) acknowledge or respond.
    • Use Case: System-wide notifications or alerts.
    • Agents Involved: Supervisor, Worker1, Worker2
    • Flow:
      • Supervisor assigns tasks to Worker1 and Worker2.
      • Workers execute tasks and report progress back to Supervisor.
    • Use Case: Task delegation in structured organizations.
    • Agents Involved: Publisher, Subscriber1, Topic
    • Flow:
      • Publisher publishes an event or message to a Topic.
      • Subscriber1, who is subscribed to the Topic, receives the event.
    • Use Case: Decoupled systems where publishers and subscribers do not need direct knowledge of each other.
    • Agents Involved: TriggerEvent, ReactiveAgent, NextStep
    • Flow:
      • An event occurs (TriggerEvent).
      • ReactiveAgent detects the event and acts.
      • The action leads to the NextStep in the process.
    • Use Case: Systems that need to respond to asynchronous events or changes in the environment.

    Since, we now understand the basic communication types. Let us understand the AutoGen generated first code & the last code (That satisfies our need) –

    # filename: simple_snake.py (Generated by AutoGen)
    
    import pygame
    import time
    import random
     
    snake_speed = 15
     
    # Window color
    white = pygame.Color(255, 255, 255)
     
    # Snake color
    green = pygame.Color(0, 255, 0)
     
    snake_position = [100, 50]
     
    # defining first 4 blocks 
    # of snake body
    snake_body = [ [100, 50], 
                   [90, 50],
                   [80, 50],
                   [70, 50]
                ]
    # fruit position
    fruit_position = [random.randrange(1, (1000//10)) * 10, 
                      random.randrange(1, (600//10)) * 10]
    fruit_spawn = True
     
    direction = 'RIGHT'
    change_to = direction
     
    score = 0
     
    # Initialising pygame
    pygame.init()
     
    # Initialise game window
    win = pygame.display.set_mode((1000, 600))
    pygame.display.set_caption("Snake game for kids")
     
    # FPS (frames per second) controller
    fps_controller = pygame.time.Clock()
     
      
    while True:
        # handling key events
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
                if event.key == pygame.K_UP:
                    change_to = 'UP'
                if event.key == pygame.K_DOWN:
                    change_to = 'DOWN'
                if event.key == pygame.K_LEFT:
                    change_to = 'LEFT'
                if event.key == pygame.K_RIGHT:
                    change_to = 'RIGHT'
    
        # If two keys pressed simultaneously
        # we don't want snake to move into two
        # directions simultaneously
        if change_to == 'UP' and direction != 'DOWN':
            direction = 'UP'
        if change_to == 'DOWN' and direction != 'UP':
            direction = 'DOWN'
        if change_to == 'LEFT' and direction != 'RIGHT':
            direction = 'LEFT'
        if change_to == 'RIGHT' and direction != 'LEFT':
            direction = 'RIGHT'
     
        # Moving the snake
        if direction == 'UP':
            snake_position[1] -= 10
        if direction == 'DOWN':
            snake_position[1] += 10
        if direction == 'LEFT':
            snake_position[0] -= 10
        if direction == 'RIGHT':
            snake_position[0] += 10
     
        # Snake body growing mechanism
        # if fruits and snakes collide then scores
        # will increase by 10
        snake_body.insert(0, list(snake_position))
        if snake_position[0] == fruit_position[0] and snake_position[1] == fruit_position[1]:
            score += 10
            fruit_spawn = False
        else:
            snake_body.pop()
             
        if not fruit_spawn:
            fruit_position = [random.randrange(1, (1000//10)) * 10, 
                              random.randrange(1, (600//10)) * 10]
             
        fruit_spawn = True
        win.fill(white)
        
        for pos in snake_body:
            pygame.draw.rect(win, green, pygame.Rect(pos[0], pos[1], 10, 10))
        pygame.draw.rect(win, white, pygame.Rect(
        fruit_position[0], fruit_position[1], 10, 10))
     
        # Game Over conditions
        if snake_position[0] < 0 or snake_position[0] > 1000-10:
            break
        if snake_position[1] < 0 or snake_position[1] > 600-10:
            break
     
        # Touching the snake body
        for block in snake_body[1:]:
            if snake_position[0] == block[0] and snake_position[1] == block[1]:
                break
        
        # refresh game screen
        pygame.display.update()
    
        # Frame Per Second /Refresh rate
        fps_controller.tick(snake_speed)
    
    # displaying final score after game over
    print(f"You scored {score} in the game.")

    Now, let us remember the prompt refinement that we’ve done (We’ve already posted that in our last post as well) –

    I want to place the snake within the boundary of the visual wall & it should bounce each wall & then hit the target. And, it should increase the size. And, every time when it hits the target it will increase the score by 10. And, the application will run for 2 minutes, which will be displayed on the screen. This will exit once 2 minute is over.
    # filename: simple_snake.py (Generated by AutoGen)
    
    import pygame
    import time
    import random
    import math
    
    pygame.init()
    
    white = (255, 255, 255)
    yellow = (255, 255, 102)
    green = (0, 255, 0)
    red = (255, 0, 0)
    black = (0, 0, 0)
    blue = (0, 0, 255)
    
    dis_width = 800
    dis_height = 600
    
    dis = pygame.display.set_mode((dis_width, dis_height))
    pygame.display.set_caption('Snake Game')
    
    clock = pygame.time.Clock()
    snake_block = 10
    snake_speed = 30
    font_style = pygame.font.SysFont(None, 50)
    score_font = pygame.font.SysFont(None, 35)
    
    def our_snake(snake_block, snake_List):
        for x in snake_List:
            pygame.draw.rect(dis, green, [x[0], x[1], snake_block, snake_block])
    
    def message(msg,color):
        mesg = font_style.render(msg, True, color)
        dis.blit(mesg, [dis_width / 3, dis_height / 3])
    
    def gameLoop():  # creating a function
        game_over = False
        game_close = False
    
        # snake starting coordinates
        x1 = dis_width / 2
        y1 = dis_height / 2
    
        # snake initial movement direction
        x1_change = 0
        y1_change = 0
    
        # initialize snake length and list of coordinates
        snake_List = []
        Length_of_snake = 1
    
        # random starting point for the food
        foodx = round(random.randrange(0, dis_width - snake_block) / 10.0) * 10.0
        foody = round(random.randrange(0, dis_height - snake_block) / 10.0) * 10.0
    
        # initialize score
        score = 0
    
        # store starting time
        start_time = time.time()
    
        while not game_over:
    
            # Remaining time
            elapsed_time = time.time() - start_time
            remaining_time = 120 - elapsed_time  # 2 minutes game
            if remaining_time <= 0:
                game_over = True
    
            # event handling loop
            for event in pygame.event.get():
                if event.type == pygame.QUIT:
                    game_over = True  # when closing window
                if event.type == pygame.MOUSEBUTTONUP:
                    # get mouse click coordinates
                    pos = pygame.mouse.get_pos()
    
                    # calculate new direction vector from snake to click position
                    x1_change = pos[0] - x1
                    y1_change = pos[1] - y1
    
                    # normalize direction vector
                    norm = math.sqrt(x1_change ** 2 + y1_change ** 2)
                    if norm != 0:
                        x1_change /= norm
                        y1_change /= norm
    
                    # multiply direction vector by step size
                    x1_change *= snake_block
                    y1_change *= snake_block
    
            x1 += x1_change
            y1 += y1_change
            dis.fill(white)
            pygame.draw.rect(dis, red, [foodx, foody, snake_block, snake_block])
            pygame.draw.rect(dis, green, [x1, y1, snake_block, snake_block])
            snake_Head = []
            snake_Head.append(x1)
            snake_Head.append(y1)
            snake_List.append(snake_Head)
            if len(snake_List) > Length_of_snake:
                del snake_List[0]
    
            our_snake(snake_block, snake_List)
    
            # Bounces the snake back if it hits the edge
            if x1 < 0 or x1 > dis_width:
                x1_change *= -1
            if y1 < 0 or y1 > dis_height:
                y1_change *= -1
    
            # Display score
            value = score_font.render("Your Score: " + str(score), True, black)
            dis.blit(value, [0, 0])
    
            # Display remaining time
            time_value = score_font.render("Remaining Time: " + str(int(remaining_time)), True, blue)
            dis.blit(time_value, [0, 30])
    
            pygame.display.update()
    
            # Increase score and length of snake when snake gets the food
            if abs(x1 - foodx) < snake_block and abs(y1 - foody) < snake_block:
                foodx = round(random.randrange(0, dis_width - snake_block) / 10.0) * 10.0
                foody = round(random.randrange(0, dis_height - snake_block) / 10.0) * 10.0
                Length_of_snake += 1
                score += 10
    
            # Snake movement speed
            clock.tick(snake_speed)
    
        pygame.quit()
        quit()
    
    gameLoop()
    

    Now, let us understand the difference here –

    The first program is a snake game controlled by arrow keys that end if the Snake hits a wall or itself. The second game uses mouse clicks for control, bounces off walls instead of ending, includes a 2-minute timer, and displays the remaining time.

    So, we’ve done it. 🙂

    You can find the detailed code in the following Github link.


    I’ll bring some more exciting topics in the coming days from the Python verse.

    Till then, Happy Avenging! 🙂

    Building solutions using LLM AutoGen in Python – Part 1

    Today, I’ll be publishing a series of posts on LLM agents and how they can help you improve your delivery capabilities for various tasks.

    Also, we’re providing the demo here –

    Isn’t it exciting?


    The application will interact with the AutoGen agents, use underlying Open AI APIs to follow the instructions, generate the steps, and then follow that path to generate the desired code. Finally, it will execute the generated scripts if the first outcome of the demo satisfies users.


    Let us understand some of the key snippets –

    # Create the assistant agent
    assistant = autogen.AssistantAgent(
        name="AI_Assistant",
        llm_config={
            "config_list": config_list,
        }
    )

    Purpose: This line creates an AI assistant agent named “AI_Assistant”.

    Function: It uses a language model configuration provided in config_list to define how the assistant behaves.

    Role: The assistant serves as the primary agent who will coordinate with other agents to solve problems.

    user_proxy = autogen.UserProxyAgent(
        name="Admin",
        system_message=templateVal_1,
        human_input_mode="TERMINATE",
        max_consecutive_auto_reply=10,
        is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
        code_execution_config={
            "work_dir": WORK_DIR,
            "use_docker": False,
        },
    )

    Purpose: This code creates a user proxy agent named “Admin”.

    Function:

    • System Message: Uses templateVal_1 as its initial message to set the context.
    • Human Input Mode: Set to "TERMINATE", meaning it will keep interacting until a termination condition is met.
    • Auto-Reply Limit: Can automatically reply up to 10 times without human intervention.
    • Termination Condition: A message is considered a termination message if it ends with the word “TERMINATE”.
    • Code Execution: Configured to execute code in the directory specified by WORK_DIR without using Docker.

    Role: Acts as an intermediary between the user and the assistant, handling interactions and managing the conversation flow.

    engineer = autogen.AssistantAgent(
        name="Engineer",
        llm_config={
            "config_list": config_list,
        },
        system_message=templateVal_2,
    )

    Purpose: Creates an assistant agent named “Engineer”.

    Function: Uses templateVal_2 as its system message to define its expertise in engineering matters.

    Role: Specializes in technical and engineering aspects of the problem.

    game_designer = autogen.AssistantAgent(
        name="GameDesigner",
        llm_config={
            "config_list": config_list,
        },
        system_message=templateVal_3,
    )

    Purpose: Creates an assistant agent named “GameDesigner”.

    Function: Uses templateVal_3 to set its focus on game design.

    Role: Provides insights and solutions related to game design aspects.

    planner = autogen.AssistantAgent(
        name="Planer",
        llm_config={
            "config_list": config_list,
        },
        system_message=templateVal_4,
    )

    Purpose: Creates an assistant agent named “Planer” (likely intended to be “Planner”).

    Function: Uses templateVal_4 to define its role in planning.

    Role: Responsible for organizing and planning tasks to solve the problem.

    critic = autogen.AssistantAgent(
        name="Critic",
        llm_config={
            "config_list": config_list,
        },
        system_message=templateVal_5,
    )

    Purpose: Creates an assistant agent named “Critic”.

    Function: Uses templateVal_5 to set its function as a critic.

    Role: Provide feedback, critique solutions, and help improve the overall response.

    logging.basicConfig(level=logging.ERROR)
    logger = logging.getLogger(__name__)

    Purpose: Configures the logging system.

    Function: Sets the logging level to only capture error messages to avoid cluttering the output.

    Role: Helps in debugging by capturing and displaying error messages.

    def buildAndPlay(self, inputPrompt):
        try:
            user_proxy.initiate_chat(
                assistant,
                message=f"We need to solve the following problem: {inputPrompt}. "
                        "Please coordinate with the admin, engineer, game_designer, planner and critic to provide a comprehensive solution. "
            )
    
            return 0
        except Exception as e:
            x = str(e)
            print('Error: <<Real-time Translation>>: ', x)
    
            return 1

    Purpose: Defines a method to initiate the problem-solving process.

    Function:

    • Parameters: Takes inputPrompt, which is the problem to be solved.
    • Action:
      • Calls user_proxy.initiate_chat() to start a conversation between the user proxy agent and the assistant agent.
      • Sends a message requesting coordination among all agents to provide a comprehensive solution to the problem.
    • Error Handling: If an exception occurs, it prints an error message and returns 1.

    Role: Initiates collaboration among all agents to solve the provided problem.

    Agents Setup: Multiple agents with specialized roles are created.
    Initiating Conversation: The buildAndPlay method starts a conversation, asking agents to collaborate.
    Problem Solving: Agents communicate and coordinate to provide a comprehensive solution to the input problem.
    Error Handling: The system captures and logs any errors that occur during execution.


    We’ll continue to discuss this topic in the upcoming post.

    I’ll bring some more exciting topics in the coming days from the Python verse.

    Till then, Happy Avenging! 🙂

    Demystifying Modern Data Technologies: Insights from the Global PowerBI Summit

    In an engaging session at the Global PowerBI Summit, we and our co-host delved into the evolving landscape of data technologies. Our discussion aimed to illuminate the distinctions and applications of several pivotal technologies in the data sphere, ranging from Lakehouse vs. Storage Account to the nuanced differences between Fabric Pipeline and Data Pipeline and the critical comparisons of Notebooks vs. Databricks, including their performance metrics. Furthermore, we explored the realm of model experimentation and Azure ML, shedding light on their performance benchmarks.

    • Enhanced File Previews and Transformations: The Lakehouse paradigm revolutionizes how we preview and transform files into SQL tables, offering a seamless data manipulation experience.
    • Robust Data Governance: It introduces native indexing for data lineage, PII scans, and discovery, thus laying a solid foundation for data governance.
    • Optimized Performance for Reporting: With direct lake mode, Lakehouse significantly improves performance for Power BI Reporting, catering to the needs of data analysts and business intelligence professionals.
    • Functional Restrictions: Despite its strengths, Lakehouse falls short in providing a native file download feature, demands manual refresh for new file visibility, and has limited support for file formats outside of Delta and Parquet.
    • Lakehouse distinguishes itself by being user-friendly and efficient in data uploading, albeit with slower previews. Its distinction from a Storage Account lies in these unique functionalities and user experience.
    • Ease of Data Transformation: It introduces a low-code, no-code approach with the Power Query Editor, enriching the data transformation process.
    • Advanced Monitoring Capabilities: The ability to monitor pipelines and trace lineage enhances the management and integration of fabric artifacts.
    • Artifact and Trigger Limitations: A notable drawback is the isolated nature of each pipeline artifact and the limitation to a single scheduled trigger type per pipeline.
    • Our analysis reveals that while both platforms share a user-friendly interface reminiscent of Azure’s, navigating between pipelines in Fabric requires additional steps. However, both platforms demonstrate rapid execution capabilities, with Azure slightly leading due to its unified pipeline management.
    • Comprehensive Support and Integration: Notably, Notebooks excel in providing native support for various programming and visualization packages, coupled with a direct connection to Lakehouse.
    • Collaborative Features and Efficiency: The platform encourages collaboration through real-time co-editing and optimizes resource usage by stopping clusters when not in use.
    • Cluster and Resource Management: External management of clusters and the absence of a shared folder or user notebooks present challenges in collaborative environments.
    • Our discussion highlighted that Notebooks offers a superior user interface and connectivity options despite Databricks’ having certain advantages in data processing speeds.

    Our performance analysis underscored Fabric Notebooks’ superiority in handling large datasets and running machine learning models more efficiently than Databricks, especially highlighting Lakehouse’s faster cluster initiation times and data storage efficiencies.

    • Seamless Integration and Configuration: Fabric’s integration with Lakehouse and direct pipeline connections streamline the data science workflow.
    • Graphical Interface and Focus: Fabric’s lack of a graphical interface contrasts with Azure ML’s user-friendly studio, indicating Fabric’s analytics and BI focus against Azure ML’s comprehensive experiment capabilities.
    • Our comparative performance review revealed that Fabric excels in dataset loading and model execution speeds, offering significant advantages over Azure ML.

    Our Global PowerBI Summit session aimed to demystify the complexities of modern data technologies, providing attendees with clear, actionable insights. Our collaborative presentation underscored the importance of understanding each technology’s strengths and limitations, empowering data professionals to make informed decisions in their projects. The dynamic interplay between these technologies illustrates the vibrant and evolving nature of the data landscape, promising exciting possibilities for innovation and efficiency in data management and analysis.

    These stats were taken during the early release of the product. However, there is a continuous improvement of this product. Hence, we need to revisit this after a period of some time.

    Text2SQL Data Extractor (T2SDE) using Python & Open AI LLM

    Today, I will share a new post that will contextualize the source files & then read the data into the pandas data frame, and then dynamically create the SQL & execute it. Then, fetch the data from the sources based on the query generated dynamically. This project is for the advanced Python developer and data Science Newbie.

    In this post, I’ve directly subscribed to OpenAI & I’m not using OpenAI from Azure. However, I’ll explore that in the future as well.

    Before I explain the process to invoke this new library, why not view the demo first & then discuss it?

    Demo

    Let us look at the flow diagram as it captures the sequence of events that unfold as part of the process.


    The application will take the metadata captured from source data dynamically. It blends the metadata and enhances the prompt to pass to the Flask server. The Flask server has all the limits of contexts.

    Once the application receives the correct generated SQL, it will then apply the SQL using the SQLAlchemy package to get the desired results.

    The following are the important packages that are essential to this project –

    pip install openai==1.6.1
    pip install pandas==2.1.4
    pip install Flask==3.0.0
    pip install SQLAlchemy==2.0.23

    We’ll have both the server and the main application. Today, we’ll be going in reverse mode. We first discuss the main script & then explain all the other class scripts.

    • 1_invokeSQLServer.py (This is the main calling Python script to invoke the OpenAI-Server.)

    Please find some of the key snippet from this discussion –

    @app.route('/message', methods=['POST'])
    def message():
        input_text = request.json.get('input_text', None)
        session_id = request.json.get('session_id', None)
    
        print('*' * 240)
        print('User Input:')
        print(str(input_text))
        print('*' * 240)
    
        # Retrieve conversation history from the session or database
        conversation_history = session.get(session_id, [])
    
        # Add the new message to the conversation history
        conversation_history.append(input_text)
    
        # Call OpenAI API with the updated conversation
        response = client.with_options(max_retries=0).chat.completions.create(
            messages=[
                {
                    "role": "user",
                    "content": input_text,
                }
            ],
            model=cf.conf['MODEL_NAME'],
        )
    
        # Extract the content from the first choice's message
        chat_response = response.choices[0].message.content
        print('*' * 240)
        print('Resposne::')
        print(chat_response)
        print('*' * 240)
    
        conversation_history.append(chat_response)
    
        # Store the updated conversation history in the session or database
        session[session_id] = conversation_history
    
        return chat_response

    This code defines a web application route that handles POST requests sent to the /message endpoint:

    1. Route Declaration: The @app.route('/message', methods=['POST']) part specifies that the function message() is executed when the server receives a POST request at the /message URL.
    2. Function Definition: Inside the message() function:
      • It retrieves two pieces of data from the request’s JSON body: input_text (the user’s input message) and session_id (a unique identifier for the user’s session).
      • It prints the user’s input message, surrounded by lines of asterisks for emphasis.
    3. Conversation History Management:
      • The code retrieves the conversation history associated with the given session_id. This history is a list of messages.
      • It then adds the new user message (input_text) to this conversation history.
    4. OpenAI API Call:
      • The function makes a call to the OpenAI API, passing the user’s message. It specifies not to retry the request if it fails (max_retries=0).
      • The model used for the OpenAI API call is taken from some configurations (cf.conf['MODEL_NAME']).
    5. Processing API Response:
      • The response from the OpenAI API is processed to extract the content of the chat response.
      • This chat response is printed.
    6. Updating Conversation History:
      • The chat response is added to the conversation history.
      • The updated conversation history is then stored back in the session or database, associated with the session_id.
    7. Returning the Response: Finally, the function returns the chat response.

    • clsDynamicSQLProcess.py (This Python class generates the SQL & then executes the flask server to invoke the OpenAI-Server.)

    Now, let us understand the few important piece of snippet –

    def text2SQLBegin(self, DBFileNameList, fileDBPath, srcQueryPrompt, joinCond, debugInd='N'):
    
            question = srcQueryPrompt
            create_table_statement = ''
            jStr = ''
    
            print('DBFileNameList::', DBFileNameList)
            print('prevSessionDBFileNameList::', self.prevSessionDBFileNameList)
    
            if set(self.prevSessionDBFileNameList) == set(DBFileNameList):
                self.flag = 'Y'
            else:
                self.flag = 'N'
    
            if self.flag == 'N':
    
                for i in DBFileNameList:
                    DBFileName = i
    
                    FullDBname = fileDBPath + DBFileName
                    print('File: ', str(FullDBname))
    
                    tabName, _ = DBFileName.split('.')
    
                    # Reading the source data
                    df = pd.read_csv(FullDBname)
    
                    # Convert all string columns to lowercase
                    df = df.apply(lambda x: x.str.lower() if x.dtype == "object" else x)
    
                    # Convert DataFrame to SQL table
                    df.to_sql(tabName, con=engine, index=False)
    
                    # Create a MetaData object and reflect the existing database
                    metadata = MetaData()
                    metadata.reflect(bind=engine)
    
                    # Access the 'users' table from the reflected metadata
                    table = metadata.tables[tabName]
    
                    # Generate the CREATE TABLE statement
                    create_table_statement = create_table_statement + str(CreateTable(table)) + '; \n'
    
                    tabName = ''
    
                for joinS in joinCond:
                    jStr = jStr + joinS + '\n'
    
                self.prevSessionDBFileNameList = DBFileNameList
                self.prev_create_table_statement = create_table_statement
    
                masterSessionDBFileNameList = self.prevSessionDBFileNameList
                mast_create_table_statement = self.prev_create_table_statement
    
            else:
                masterSessionDBFileNameList = self.prevSessionDBFileNameList
                mast_create_table_statement = self.prev_create_table_statement
    
            inputPrompt = (templateVal_1 + mast_create_table_statement + jStr + templateVal_2).format(question=question)
    
            if debugInd == 'Y':
                print('INPUT PROMPT::')
                print(inputPrompt)
    
            print('*' * 240)
            print('Find the Generated SQL:')
            print()
    
            DBFileNameList = []
            create_table_statement = ''
    
            return inputPrompt
    1. Function Overview: The text2SQLBegin function processes a list of database file names (DBFileNameList), a file path (fileDBPath), a query prompt (srcQueryPrompt), join conditions (joinCond), and a debug indicator (debugInd) to generate SQL commands.
    2. Initial Setup: It starts by initializing variables for the question, the SQL table creation statement, and a string for join conditions.
    3. Debug Prints: The function prints the current and previous session database file names for debugging purposes.
    4. Flag Setting: A flag is set to ‘Y’ if the current session’s database file names match the previous session’s; otherwise, it’s set to ‘N’.
    5. Processing New Session Data: If the flag is ‘N’, indicating new session data:
      • For each database file, it reads the data, converts string columns to lowercase, and creates a corresponding SQL table in a database using the pandas library.
      • Metadata is generated for each table and a CREATE TABLE SQL statement is created.
    6. Join Conditions and Statement Aggregation: Join conditions are concatenated, and previous session information is updated with the current session’s data.
    7. Handling Repeated Sessions: If the session data is repeated (flag is ‘Y’), it uses the previous session’s SQL table creation statements and database file names.
    8. Final Input Prompt Creation: It constructs the final input prompt by combining template values with the create table statement, join conditions, and the original question.
    9. Debug Printing: If debug mode is enabled, it prints the final input prompt.
    10. Conclusion: The function clears the DBFileNameList and create_table_statement variables, and returns the constructed input prompt.
      def text2SQLEnd(self, srcContext, debugInd='N'):
          url = self.url
    
          payload = json.dumps({"input_text": srcContext,"session_id": ""})
          headers = {'Content-Type': 'application/json', 'Cookie': cf.conf['HEADER_TOKEN']}
    
          response = requests.request("POST", url, headers=headers, data=payload)
    
          return response.text

    The text2SQLEnd function sends an HTTP POST request to a specified URL and returns the response. It takes two parameters: srcContext which contains the input text, and an optional debugInd for debugging purposes. The function constructs the request payload by converting the input text and an empty session ID to JSON format. It sets the request headers, including a content type of ‘application/json’ and a token from the configuration file. The function then sends the POST request using the requests library and returns the text content of the response.

      def sql2Data(self, srcSQL):
          # Executing the query on top of your data
          resultSQL = pd.read_sql_query(srcSQL, con=engine)
    
          return resultSQL

    The sql2Data function is designed to execute a SQL query on a database and return the result. It takes a single parameter, srcSQL, which contains the SQL query to be executed. The function uses the pandas library to run the provided SQL query (srcSQL) against a database connection (engine). It then returns the result of this query, which is typically a DataFrame object containing the data retrieved from the database.

    def genData(self, srcQueryPrompt, fileDBPath, DBFileNameList, joinCond, debugInd='N'):
        try:
            authorName = self.authorName
            website = self.website
            var = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
    
            print('*' * 240)
            print('SQL Start Time: ' + str(var))
            print('*' * 240)
    
            print('*' * 240)
            print()
    
            if debugInd == 'Y':
                print('Author Name: ', authorName)
                print('For more information, please visit the following Website: ', website)
                print()
    
                print('*' * 240)
            print('Your Data for Retrieval:')
            print('*' * 240)
    
            if debugInd == 'Y':
    
                print()
                print('Converted File to Dataframe Sample:')
                print()
    
            else:
                print()
    
            context = self.text2SQLBegin(DBFileNameList, fileDBPath, srcQueryPrompt, joinCond, debugInd)
            srcSQL = self.text2SQLEnd(context, debugInd)
    
            print(srcSQL)
            print('*' * 240)
            print()
            resDF = self.sql2Data(srcSQL)
    
            print('*' * 240)
            print('SQL End Time: ' + str(var))
            print('*' * 240)
    
            return resDF
    
        except Exception as e:
            x = str(e)
            print('Error: ', x)
    
            df = pd.DataFrame()
    
            return df
    1. Initialization and Debug Information: The function begins by initializing variables like authorName, website, and a timestamp (var). It then prints the start time of the SQL process. If the debug indicator (debugInd) is ‘Y’, it prints additional information like the author’s name and website.
    2. Generating SQL Context and Query: The function calls text2SQLBegin with various parameters (file paths, database file names, query prompt, join conditions, and the debug indicator) to generate an SQL context. Then it calls text2SQLEnd with this context and the debug indicator to generate the actual SQL query.
    3. Executing the SQL Query: It prints the generated SQL query for visibility, especially in debug mode. The query is then executed by calling sql2Data, which returns the result as a data frame (resDF).
    4. Finalization and Error Handling: After executing the query, it prints the SQL end time. In case of any exceptions during the process, it catches the error, prints it, and returns an empty DataFrame.
    5. Return Value: The function returns the DataFrame (resDF) containing the results of the executed SQL query. If an error occurs, it returns an empty DataFrame instead.

    Let us explore the directory structure starting from the parent to some of the important child folder should look like this –

    Let us understand the important screenshots of this entire process –


    So, finally, we’ve done it.

    You will get the complete codebase in the following GitHub link.

    I’ll bring some more exciting topics in the coming days from the Python verse. Please share & subscribe to my post & let me know your feedback.

    Till then, Happy Avenging! 🙂

    Exploring the new Polars library in Python

    Today, I will present some valid Python packages where you can explore most of the complex SQLs by using this new package named “Polars,” which can be extremely handy on many occasions.

    This post will be short posts where I’ll prepare something new on LLMs for the upcoming posts for the next month.

    Why not view the demo before going through it?


    Demo
    pip install polars
    pip install pandas

    Let us understand the key class & snippets.

    • clsConfigClient.py (Key entries that will be discussed later)
    ################################################
    #### Written By: SATYAKI DE                 ####
    #### Written On:  15-May-2020               ####
    #### Modified On: 28-Oct-2023               ####
    ####                                        ####
    #### Objective: This script is a config     ####
    #### file, contains all the keys for        ####
    #### personal OpenAI-based MAC-shortcuts    ####
    #### enable bot.                            ####
    ####                                        ####
    ################################################
    
    import os
    import platform as pl
    
    class clsConfigClient(object):
        Curr_Path = os.path.dirname(os.path.realpath(__file__))
    
        os_det = pl.system()
        if os_det == "Windows":
            sep = '\\'
        else:
            sep = '/'
    
        conf = {
            'APP_ID': 1,
            'ARCH_DIR': Curr_Path + sep + 'arch' + sep,
            'LOG_PATH': Curr_Path + sep + 'log' + sep,
            'DATA_PATH': Curr_Path + sep + 'data' + sep,
            'TEMP_PATH': Curr_Path + sep + 'temp' + sep,
            'OUTPUT_DIR': 'model',
            'APP_DESC_1': 'Polars Demo!',
            'DEBUG_IND': 'Y',
            'INIT_PATH': Curr_Path,
            'TITLE': "Polars Demo!",
            'PATH' : Curr_Path,
            'OUT_DIR': 'data',
            'MERGED_FILE': 'mergedFile.csv',
            'ACCT_FILE': 'AccountAddress.csv',
            'ORDER_FILE': 'Orders.csv',
            'CUSTOMER_FILE': 'CustomerDetails.csv',
            'STATE_CITY_WISE_REPORT_FILE': 'StateCityWiseReport.csv'
        }
    
    • clsSQL.py (Main class file that contains how to use the SQL)
    #####################################################
    #### Written By: SATYAKI DE                      ####
    #### Written On: 27-May-2023                     ####
    #### Modified On 28-Oct-2023                     ####
    ####                                             ####
    #### Objective: This is the main calling         ####
    #### python class that will invoke the           ####
    #### Polar class, which will enable SQL          ####
    #### capabilitites.                              ####
    ####                                             ####
    #####################################################
    
    import polars as pl
    import os
    from clsConfigClient import clsConfigClient as cf
    import pandas as p
    
    ###############################################
    ###           Global Section                ###
    ###############################################
    
    # Disbling Warning
    def warn(*args, **kwargs):
        pass
    
    import warnings
    warnings.warn = warn
    
    ###############################################
    ###    End of Global Section                ###
    ###############################################
    
    class clsSQL:
        def __init__(self):
            self.acctFile = cf.conf['ACCT_FILE']
            self.orderFile = cf.conf['ORDER_FILE']
            self.stateWiseReport = cf.conf['STATE_CITY_WISE_REPORT_FILE']
            self.custFile = cf.conf['CUSTOMER_FILE']
            self.dataPath = cf.conf['DATA_PATH']
    
        def execSQL(self):
            try:
                dataPath = self.dataPath
                acctFile = self.acctFile
                orderFile = self.orderFile
                stateWiseReport = self.stateWiseReport
                custFile = self.custFile
    
                fullAcctFile = dataPath + acctFile
                fullOrderFile = dataPath + orderFile
                fullStateWiseReportFile = dataPath + stateWiseReport
                fullCustomerFile = dataPath + custFile
    
                ctx = pl.SQLContext(accountMaster = pl.scan_csv(fullAcctFile),
                orderMaster = pl.scan_csv(fullOrderFile),
                stateMaster = pl.scan_csv(fullStateWiseReportFile))
    
                querySQL = """
                SELECT orderMaster.order_id,
                orderMaster.total,
                stateMaster.state,
                accountMaster.Acct_Nbr,
                accountMaster.Name,
                accountMaster.Email,
                accountMaster.user_id,
                COUNT(*) TotalCount
                FROM orderMaster
                JOIN stateMaster USING (city)
                JOIN accountMaster USING (user_id)
                ORDER BY stateMaster.state
                """
    
                res = ctx.execute(querySQL, eager=True)
                res_Pandas = res.to_pandas()
    
                print('Result:')
                print(res_Pandas)
                print(type(res_Pandas))
    
                ctx_1 = pl.SQLContext(customerMaster = pl.scan_csv(fullCustomerFile),
                tempMaster=pl.from_pandas(res_Pandas))
    
                querySQL_1 = """
                SELECT tempMaster.order_id,
                tempMaster.total,
                tempMaster.state,
                tempMaster.Acct_Nbr,
                tempMaster.Name,
                tempMaster.Email,
                tempMaster.TotalCount,
                tempMaster.user_id,
                COUNT(*) OVER(PARTITION BY tempMaster.state ORDER BY tempMaster.state, tempMaster.Acct_Nbr) StateWiseCount,
                MAX(tempMaster.Acct_Nbr) OVER(PARTITION BY tempMaster.state ORDER BY tempMaster.state, tempMaster.Acct_Nbr) MaxAccountByState,
                MIN(tempMaster.Acct_Nbr) OVER(PARTITION BY tempMaster.state ORDER BY tempMaster.state, tempMaster.Acct_Nbr) MinAccountByState,
                CASE WHEN tempMaster.total < 70 THEN 'SILVER' ELSE 'GOLD' END CategoryStat,
                SUM(customerMaster.Balance) OVER(PARTITION BY tempMaster.state) SumBalance
                FROM tempMaster
                JOIN customerMaster USING (user_id)
                ORDER BY tempMaster.state
                """
    
                res_1 = ctx_1.execute(querySQL_1, eager=True)
    
                finDF = res_1.to_pandas()
    
                print('Result 2:')
                print(finDF)
    
                return 0
            except Exception as e:
                discussedTopic = []
                x = str(e)
                print('Error: ', x)
    
                return 1
    

    If we go through some of the key lines, we will understand how this entire package works.

    But, before that, let us understand the source data –

    Let us understand the steps –

    1. Join orderMaster, stateMaster & accountMaster and fetch the selected attributes. Store this in a temporary data frame named tempMaster.
    2. Join tempMaster & customerMaster and fetch the relevant attributes with some more aggregation, which is required for the business KPIs.
    ctx = pl.SQLContext(accountMaster = pl.scan_csv(fullAcctFile),
    orderMaster = pl.scan_csv(fullOrderFile),
    stateMaster = pl.scan_csv(fullStateWiseReportFile))

    The above method will create three temporary tables by reading the source files – AccountAddress.csv, Orders.csv & StateCityWiseReport.csv.

    And, let us understand the supported SQLs –

    SELECT  orderMaster.order_id,
            orderMaster.total,
            stateMaster.state,
            accountMaster.Acct_Nbr,
            accountMaster.Name,
            accountMaster.Email,
            accountMaster.user_id,
            COUNT(*) TotalCount
    FROM orderMaster
    JOIN stateMaster USING (city)
    JOIN accountMaster USING (user_id)
    ORDER BY stateMaster.state

    In this step, we’re going to store the output of the above query into a temporary view named – tempMaster data frame.

    Since this is a polar data frame, we’re converting it to the pandas data frame.

    res_Pandas = res.to_pandas()

    Finally, let us understand the next part –

    ctx_1 = pl.SQLContext(customerMaster = pl.scan_csv(fullCustomerFile),
    tempMaster=pl.from_pandas(res_Pandas))

    In the above section, one source is getting populated from the CSV file, whereas the other source is feeding from a pandas data frame populated in the previous step.

    Now, let us understand the SQL supported by this package, which is impressive –

    SELECT  tempMaster.order_id,
            tempMaster.total,
            tempMaster.state,
            tempMaster.Acct_Nbr,
            tempMaster.Name,
            tempMaster.Email,
            tempMaster.TotalCount,
            tempMaster.user_id,
            COUNT(*) OVER(PARTITION BY tempMaster.state ORDER BY tempMaster.state, tempMaster.Acct_Nbr) StateWiseCount,
            MAX(tempMaster.Acct_Nbr) OVER(PARTITION BY tempMaster.state ORDER BY tempMaster.state, tempMaster.Acct_Nbr) MaxAccountByState,
            MIN(tempMaster.Acct_Nbr) OVER(PARTITION BY tempMaster.state ORDER BY tempMaster.state, tempMaster.Acct_Nbr) MinAccountByState,
            CASE WHEN tempMaster.total < 70 THEN 'SILVER' ELSE 'GOLD' END CategoryStat,
            SUM(customerMaster.Balance) OVER(PARTITION BY tempMaster.state) SumBalance
    FROM tempMaster
    JOIN customerMaster USING (user_id)
    ORDER BY tempMaster.state

    As you can see it has the capability of all the advanced analytics SQL using partitions, and CASE statements.

    The only problem with COUNT(*) with the partition is not working as expected. Not sure, whether that is related to any version issues or not.

    COUNT(*) OVER(PARTITION BY tempMaster.state ORDER BY tempMaster.state, tempMaster.Acct_Nbr) StateWiseCount

    I’m trying to get more information on this. Except for this statement, everything works perfectly.

    • 1_testSQL.py (Main class file that contains how to use the SQL)
    #########################################################
    #### Written By: SATYAKI DE                          ####
    #### Written On: 27-Jun-2023                         ####
    #### Modified On 28-Oct-2023                         ####
    ####                                                 ####
    #### Objective: This is the main class that invokes  ####
    #### advanced analytic SQL in python.                ####
    ####                                                 ####
    #########################################################
    
    from clsConfigClient import clsConfigClient as cf
    import clsL as log
    import clsSQL as ccl
    
    from datetime import datetime, timedelta
    
    # Disbling Warning
    def warn(*args, **kwargs):
        pass
    
    import warnings
    warnings.warn = warn
    
    ###############################################
    ###           Global Section                ###
    ###############################################
    
    #Initiating Logging Instances
    clog = log.clsL()
    cl = ccl.clsSQL()
    
    var = datetime.now().strftime(".%H.%M.%S")
    
    documents = []
    
    ###############################################
    ###    End of Global Section                ###
    ###############################################
    def main():
        try:
            var = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
            print('*'*120)
            print('Start Time: ' + str(var))
            print('*'*120)
    
            r1 = cl.execSQL()
    
            if r1 == 0:
                print()
                print('Successfully SQL-enabled!')
            else:
                print()
                print('Failed to senable SQL!')
    
            print('*'*120)
            var1 = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
            print('End Time: ' + str(var1))
    
        except Exception as e:
            x = str(e)
            print('Error: ', x)
    
    if __name__ == '__main__':
        main()
    

    As this is extremely easy to understand & self-explanatory.

    To learn more about this package, please visit the following link.


    So, finally, we’ve done it. I know that this post is relatively smaller than my earlier post. But, I think, you can get a good hack to improve some of your long-running jobs by applying this trick.

    I’ll bring some more exciting topics in the coming days from the Python verse. Please share & subscribe to my post & let me know your feedback.

    Till then, Happy Avenging!  🙂