README.md

# Athene-V2-Agent: Surpassing GPT-4o for Tool Use And Agentic Usecases
<p align="center">
<a href="https://huggingface.co./Nexusflow" target="_blank">Nexusflow HF</a> - <a href="https://discord.gg/HDSVmNAs3y" target="_blank">Nexusflow Discord</a> 
</p>

<p align="center" width="100%">
<a><img src="agent.png" alt="NexusRaven" style="width: 40%; min-width: 300px; display: block; margin: auto;"></a>
</p>

## Introducing Athene-V2-Agent
Athene-V2-Agent is an open-source Agent LLM that surpasses the state-of-the-art in function calling and agentic capabilities. 

💪 **Versatile Agent Capability**: Athene-V2-Agent is an agent model, capable of operating in environments with deeply nested dependencies with the environment. It is capable of reasoning and doing planning for trajectories with many tool calls necessary to answer a single query.

📊 **Performance Highlights**: Athene-V2-Agent surpasses GPT-4o in single FC tasks by 18% in function calling success rates, and by 17% in Agentic success rates. 

🔧 **Generalization to the Unseen**: Athene-V2-Agent has never been trained on the functions or agentic settings used in evaluation.

## Athene-V2-Agent Model Usage

### OpenAI-Compatible FC

Athene-V2-Agent is usable in any OpenAI API-compatible environment using our VLLM docker image. This should be a simple "drop-in" replacement to any agentic or tool-use setting with our VLLM docker image.

```
docker run --name athene-v2-agent \
    --runtime nvidia --gpus '"device=0,1,2,3,4,5,6,7"' \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HUGGING_FACE_HUB_TOKEN=<secret>" \
    -p <port>:8000 \
    --ipc=host \
    ghcr.io/nexusflowai/athene-v2-vllm:latest \
    --model Nexusflow/Athene-V2-Agent \
    --dtype=auto \
    --tensor-parallel-size=8 \
    --enable-auto-tool-choice \
    --tool-call-parser Athene-V2-Agent
```

You can now submit any OpenAI-Compatible tool-use requests to the model by hitting the VLLM endpoint. Athene-V2-Agent will be able to issue tool calls that you can execute and return results for. 

### Examples

An example Weather agent for this can be found here: [Link](example/vllm_v2_weather_agent.py#L186-L193). This example includes handling Athene for queries that are answerable and not answerable by the current tools. 

An example extraction and RAG-Agent can be found here: [Link](example/vllm_v2_extraction_agent.py#L270-L284). This example includes handling RAG-based queries with a wikipedia tool.

### Prompting Tricks

1. When giving docstrings to Athene-V2-Agent, please provide well-indented, detailed, and well-written docstrings as this can help accuracy.
2. We strongly recommend using the docker image to interact with Athene-V2-Agent.
4. We strongly recommend to set sampling to False when prompting Athene-V2-Agent.
5. We strongly recommend a zero temperature. 

When handling irrelevant user queries, users have noticed that specifying a "no-op" function with arguments work best. For example, something like this might work:
```python
{
    "type": "function",
    "function" : {
      "name": "no_relevant_function",
      "description": "Call this when no other provided function can be called to answer the user query.",
      "parameters": {
        "type": "object",
        "properties": {
          "user_query_span": {
            "type": "string",
            "description": "The part of the user_query that cannot be answered by any other function calls."
          }
        },
        "required": ["user_query_span"]
      }
    }
}
```

## Contact
Please join our [Discord Channel](https://discord.gg/HDSVmNAs3y) to reach out for any issues and comments!