Introduction
In the realm of artificial intelligence, the adoption of multi-agent systems (MAS) represents a paradigm shift towards more dynamic and complex problem-solving capabilities. This blog dives into the essence of Multi Agent Systems, highlighting the necessity for such systems in today’s technological landscape and exploring the CrewAI framework as a possible solution.
We will build an Assignment Helper Multi Agent System with CrewAI which students can use to get ideas and initial drafts for their assignments. Through this demo, we will see how the framework works and its pros and cons.
Why Multi-Agent Systems?
Complex Problem Solving : MAS excel in environments where problems are too complex for a single agent. By dividing tasks among multiple agents, solutions become more feasible and efficient. This is a step-up from the standard chat completion end-point that can do one task at a time.
Agent Collaboration : Agents are able to work autonomously with the tools and knowledge they have, and also interact with each other and pass outputs back and forth, often leading to self-organization and emergent behaviours that are more than the sum of individual agents’ actions
Adaptability : Agents can quickly adapt to changes in their environment, making MAS ideal for dynamic situations.
Efficiency and Scalability : MAS can handle tasks concurrently, scaling as necessary to meet demand.
Multi-Agent systems can be used to solve a huge number of problems. Some examples of problems that current systems can solve are:
- Trip/Restaurant Meal planning
- Analysis/Research on different topics
- Designing a Course on a topic
- Code writer and Evaluator crew
In short Multi Agent Systems represent a leap towards creating intelligent systems that can mimic human teamwork and collaboration, tackling problems beyond the scope of individual agents.
CrewAI Framework
CrewAI Logo from their website
CrewAI is a recent open-source framework for building Multi-Agent Systems. It is built on top of Langchain. It stands out for a few reasons:
User-Friendly Design : CrewAI simplifies the complexity of MAS, making it accessible to engineers without sacrificing power. It does this by having a modular design that can be adapted to many problems.
Modular Components:
- ** Agents** : Perform tasks with specific roles and goals.
- ** Tasks** : Defined actions that agents execute.
- ** Crew** : The collective working towards a common objective.
- ** Tools** : Resources agents use to accomplish tasks.
Flexible Integration : Seamlessly integrates with Langchain, making it easy to use the tools defined in Langchain and use both open source or closed source models.
Communication and Coordination : The library handles information flow between agents, allowing an agent to continue forward on the work from where the last agent left it.
Versatility : Applicable across multiple domains, from marketing to project management. Once you understand the design paradigm, it is quite easy to design very different applications following the same principles.
Building an Student Assignment Helper Crew
Agents and Tasks
Now, lets test the CrewAI framework on a simple but useful task. We will build a team of agents who will help students do their assignments. The crew will get instructions from the user and go through 4 tasks :
- Research to get ideas and create an outline for the report
- Write the content of the report
- Evaluate the quality of the report and make a list of improvements (if any)
- Rewrite the report based on the feedback
This flow is shown below:
Flow for the Assignment Helper Crew
I selected a critic and improvement task as this often quite useful to improve the final outcome and also to see how this framework handles back and forth between agents.
To accomplish this task, we will have 3 agents:
- Idea Generator Agent — Will do research and create outline of the report. This is a GPT4-turbo agent and will accomplish the task to generate ideas through research tools and write the outline of the report
- Report Writer Agent — Will use the outline and its tools to write the report. This agent will be responsible for rewriting the report if its given feedback on it.
- Report Critic Agent — Will evaluate the report based on rubric and highlight improvement opportunities
The flow with the agents and their corresponding tasks is given below:
Assignment Crew with Tasks and Agents
This now highlights the basic layer of the CrewAI framework —
- Break your problem into sequential tasks — CrewAI can currently only handle sequential tasks. So if We want to have a back and forth between the writer and critic, We need to create a rewrite tasks to keep the flow sequential. This is one of the main drawback of the system right now forcing you to think how you can make your problem sequential
- Assign Agents to a task : Design LLM agents that are task specific and assign them to the task. You can use an agent for multiple tasks like We did but the framework requires you to do a one-one mapping between tasks and agents. We would love to see this framework evolve to having multiple agents collaborate on a single task
Giving tools to Agents
The Agents (LLMs) get super power through Tools. This is now a very familiar concept across many frameworks like Langchain and OpenAI. The tools are integration to external libraries or databases that agents can use to enhance their knowledge and skills.
CrewAI borrows the tools from Langchain which gives you access to many tools in Langchain, as well as a structured format to define your own tool.
For our application, we will use 3 tools:
- Search Tool — To get information from the web. For now we will use the free DuckDuckgo Search tool but can be replaced by any Search tool
- Wikidata Tool — Get information from Wikipedia
- Wikidata Tool — Another way to get structured information about a person, place or event from Wiki database
The tools will be assigned to the agents to give them the capability to do research and gather information. The final Assignment Crew Design is below:
Assignment Crew design with Tasks, Agents and Tools
Coding Time!
The Tools we are using are defined on Langchain and can be exported from there.
from langchain_community.tools.wikidata.tool import WikidataAPIWrapper, WikidataQueryRun
from langchain.tools import WikipediaQueryRun
from langchain_community.utilities import WikipediaAPIWrapper
from langchain_community.tools import DuckDuckGoSearchRun
wikipedia = WikipediaQueryRun(api_wrapper=WikipediaAPIWrapper())
wikidata = WikidataQueryRun(api_wrapper=WikidataAPIWrapper())
search_tool = DuckDuckGoSearchRun()The Agents are defined by setting their prompts and giving them access to the tools they need. You can also specify the LLM model to use for them. To improve the system performance, you would need to experiment and adjust the agent prompts.
class AssignmentHelperAgents:
def __init__(self):
self.OpenAIGPT41106 = ChatOpenAI(model_name="gpt-4-1106-preview ", temperature=0.3)
self.OpenAIGPT40125 = ChatOpenAI(model_name="gpt-4-0125-preview", temperature=0.3)
self.OpenAIGPT3Turbo= ChatOpenAI(model_name="gpt-3.5-turbo-0125", temperature=0.3)
def IdeaGeneratorAgent(self):
return Agent(
role="Report Idea Generator",
backstory=dedent("""
You help students with their research. Lets say a student wants to write a report on famous Canadians.
In this case you will search and generate ideas to explore. You will not write the report, but generate ideas
"""),
goal=dedent("""
Use the tools you have to search and generate ideas for the user.
"""),
allow_delegation=False,
verbose=True,
llm=self.OpenAIGPT40125,
tools=[search_tool, wikipedia, wikidata]
)
def ReportWriterAgent(self):
return Agent(
role="Report Writer",
backstory=dedent("""
You help students with writing reports for their projects.
You get the ideas from the Idea Generator and then use the tools you have to write the report.
If feedback is give to you on your report, then you will revise your report
"""),
goal=dedent("""
You help write the report based on the ideas generated by the Idea Generator. You will structure the report based on direction by the user.
A report should have these sections: Introduction, Body, Conclusion.
You will use the Search, Wikipedia and Wikidata tools you have to collect data to write the report. If feedback is given to you then you will use the feedback to revise the report
"""),
allow_delegation=False,
verbose=True,
llm=self.OpenAIGPT40125,
tools=[search_tool, wikipedia, wikidata]
)
def ReportEvaluatorAgent(self):
return Agent(
role="Report Evaluator",
backstory=dedent("""\
You look through the written report and identify any opportunities for improvement
"""),
goal=dedent("""\
You review the written report and check it against the rubric -
1. Does it meets the user criteria, 2. Is it coherent. 3. Is it the appropriate length.
4. Does it has an intro, body and conclusion,
5. Is it interesting, 6. Is it accurate
You can use the tools you have to check the accuracy of the report
. Based on your observation you provide opportunities for improvement or a pass
"""),
allow_delegation=True,
llm=self.OpenAIGPT40125,
verbose=True,
tools=[search_tool, wikidata, wikipedia]
)The tasks are a description of what needs to be accomplished. Agents are assigned to an individual task.
class AssignmentWritingTasks:
def generate_ideas(self, agent, task):
return Task(
description=dedent(
f"""
Users task is below. Use the search tools you have to generate ideas for the report. Only generate ideas.
{task}
"""
),
agent=agent,
)
def write_report(self, agent, task):
return Task(
description=dedent(
f"""
Use the ideas from the idea generator to now write a report for the student task below.
{task}
"""
),
agent=agent,
)
def evaluate_report(self, agent, task):
return Task(
description=dedent(
f"""
Users task is below. Review the report written and provide opportunities for improvement. Check it against your rubric
{task}
"""
),
agent=agent,
)
def rewrite_report(self, agent, task):
return Task(
description=dedent(
f"""
Users task is below. Review the initial report and the feedback from the report critic to rewrite addressing the feedback
{task}
"""
),
agent=agent,
)The overall Crew is a combination of tasks and agents. The system goes through the tasks in a sequential order.
agents = AssignmentHelperAgents()
tasks = AssignmentWritingTasks()
## Instantiate the agents
idea_generator_agent = agents.IdeaGeneratorAgent()
writer_agent = agents.ReportWriterAgent()
evaluator_agent = agents.ReportEvaluatorAgent()
## Instantiate the tasks
generate_ideas = tasks.generate_ideas(idea_generator_agent, task_description)
write_report = tasks.write_report(writer_agent, task_description)
evaluate_report = tasks.evaluate_report(evaluator_agent, task_description)
rewrite_report = tasks.rewrite_report(writer_agent, task_description)
crew_startup = Crew(
agents=[idea_generator_agent, writer_agent, evaluator_agent ],
tasks=[generate_ideas, write_report, evaluate_report, rewrite_report],
verbose=True,
full_output=True
)
result = crew_startup.kickoff()
print("\n\n########################")
print("## Run Result:")
print("########################\n")
print(result)Observations and Improvement Opportunities
I tried the above crew on a sample assignment task of:
We am a grade 4 student and We have to write about the different biomes
in the world. We want to cover the top ones and have 4-5 sentences on each.
Lets keep it interesting. Include a quiz at the endThis task kicks off the idea and outline generator which uses the Search and Wiki tools multiple times to collect data. Its output is:
[Report Idea Generator] Task output: Here are some ideas for your report on
different biomes:
1. **Introduction to Biomes**: Start with an explanation of what a biome is,
including the key characteristics that define a biome such as climate,
vegetation, and animal life. Mention how biomes can span multiple continents
and include a variety of ecosystems.
2. **Types of Biomes**: Divide this section into sub-sections
for each major type of biome, such as forests, deserts, tundras, and
aquatic biomes. For each type, describe the climate, typical plants and
animals, and any unique features.
3. **Anthropogenic Biomes**: Discuss how human activities have altered
natural biomes, leading to the creation of anthropogenic biomes. Explain
the concept of anthromes and how human interaction with the environment
shapes these biomes.
4. **The Amazon Biome**: Use the Amazon biome as a case study to
explore a specific biome in detail. Describe its diverse ecosystems,
including rainforests, savannas, and tundras. Discuss the importance of
the Amazon biome for biodiversity and the threats it faces from deforestation
and other human activities.
5. **Conservation Efforts**: Conclude your report by discussing the
importance of conserving biomes. Highlight some of the efforts being made
around the world to protect and preserve different biomes and their
unique ecosystems.
Remember to include pictures and examples to make your report more engaging.
This structure will help you cover a broad topic while providing specific
examples to illustrate your points.The report writer generates the first version of the report and the evaluator gives feedback. The feedback follows the rubric we designed. The feedback considers that the student is in Grade 4 and asks the writer to condense some things, add call to action , simplify some sections etc.
Final Answer: The report on different biomes is comprehensive and covers a
wide range of topics related to biomes, including their types, the impact
of human activities, and conservation efforts. However, there are opportunities for improvement based on the rubric criteria:
1. **User Criteria**: The report is well-tailored to a Grade 4 audience,
using clear language and examples. However, including more visual aids,
such as maps or pictures of each biome, could enhance understanding and
engagement for young readers.
2. **Coherence**: The report is coherent, with a logical flow from the
introduction of biomes, through the types of biomes, to human impacts
and conservation efforts. Each section transitions smoothly into the next.
To further improve coherence, consider adding a brief summary at the
end of each section to reinforce key points.
3. **Appropriate Length**: The report is of an appropriate length for a
comprehensive overview. However, for a Grade 4 audience, it might be
beneficial to condense some of the more complex sections, such as the
discussion on anthropogenic biomes, to maintain the reader's attention.
4. **Intro, Body, and Conclusion**: The report includes a clear introduction,
body, and conclusion. The introduction effectively sets the stage for
the discussion on biomes, and the conclusion succinctly summarizes
the importance of biomes. To strengthen the conclusion, consider adding a
call to action or suggestions for how young readers can contribute to biome
conservation.
5. **Interest**: The report is interesting, particularly the case study on
the Amazon biome, which provides a tangible example of the concepts discussed.
To increase interest, consider incorporating interactive elements or questions
that encourage the reader to think about how biomes affect their own lives or
how they can observe different biomes in their environment.
6. **Accuracy**: The report appears accurate based on the information provided.
To ensure accuracy, especially for a school report, it would be beneficial to
include a list of sources or suggest further reading for interested students.
Overall, the report is well-written and informative. By incorporating visual
aids, condensing complex sections, and adding interactive elements,
it can become an even more engaging and educational resource for young
students learning about biomes.The writer now rewrites the report incorporating feedback and does a pretty decent job.
**Introduction to Biomes**
Hello, young explorers! Today, we're going on an adventure to learn about the
amazing world of biomes. Imagine biomes as big neighborhoods where plants
and animals live. Each neighborhood has its own weather, plants, and animals
that make it special. Just like your neighborhood is different from one in
another country, biomes are different all around the world!
**What Are Biomes?**
Biomes are big areas on Earth with similar weather, plants, and animals.
There are many types, like forests, deserts, oceans, and even icy places
like the Arctic! Each biome is a home where living things have everything
they need to survive.
**Exploring Different Biomes**
- **Forests**: These are like giant gardens, full of trees, bushes, and lots
of animals. There are tropical rainforests that are very warm and wet,
and there are also cooler forests like the ones we might go camping in.
- **Deserts**: Imagine a place with lots of sand and very little rain.
Deserts can be hot during the day but cold at night. Cacti and camels love it
here!
- **Oceans**: The biggest biome is under the water! Oceans are full of fish,
whales, and beautiful corals. It's like a whole other world down there.
- **Arctic**: Brrr! These are the coldest places on Earth, where it's icy and
snowy. Polar bears and penguins are some of the animals tough enough to live
here.
**Our Impact on Biomes**
Sometimes, people can change biomes without meaning to, like cutting down too
many trees or polluting the water. This can make it hard for the plants and
animals to live there. But the good news is, we can help protect their homes
by being careful about how we treat nature.
**Let's Help Our Biome Friends!**
Did you know you can be a hero for biomes? Here are some ways to help:
- Recycle and reuse things instead of throwing them away.
- Learn more about different biomes and share what you know with friends and
family.
- Plant trees and take care of plants and animals in your neighborhood.
**Conclusion: Our Biome Adventure**
Wow, we've learned so much about biomes today! Remember, biomes are like big
neighborhoods for plants and animals, and they're all different and special.
By taking care of our planet, we make sure these neighborhoods stay happy and
healthy for all living things. What can you do today to help our biome
friends?
**Fun Activity**
Try drawing a picture of your favorite biome and the animals that live there.
Or, with a parent's help, look up more cool facts about that biome online!My experience showed that CrewAI is a good tool to start building multi agent pipelines. It has good integration with Langchain and offers a clean framework that is easy to understand and refine. But some opportunities remain. These include, ability to define more complex chains involving conditional loops and more back and forth. We will be exploring CrewAI with Langgraph next to see if this helps. Also We would like to build pipelines where multiple agents can work together.
Conclusion
The development of multi-agent systems (MAS) marks a big step in artificial intelligence, moving us towards solving more complex problems by allowing multiple AI agents to work together. As we explore what systems like CrewAI can do, we see a future where these systems can change many areas, including healthcare, smart cities, and education, by offering smart, collaborative solutions. This future is about making groups of AI agents work smarter together, tackling challenges no single AI could handle alone.
As we look forward, improving how these agents work together on complex tasks and making them better at adapting to changes are key areas for growth. The progress in MAS will likely lead to new discoveries and ways to use AI, making our daily lives and the technology we use smarter and more connected. Imagine a world where AI systems can not only help humans but also collaborate among themselves, creating a network of intelligence across different fields and industries. This is the exciting direction in which MAS is headed.