| | --- |
| | language: |
| | - en |
| | license: gemma |
| | library_name: transformers |
| | tags: |
| | - function-calling |
| | - agent-routing |
| | - multi-agent |
| | - lora |
| | - peft |
| | - gemma |
| | - functiongemma |
| | - customer-support |
| | - e-commerce |
| | base_model: google/functiongemma-270m-it |
| | datasets: |
| | - scionoftech/functiongemma-e-commerce-dataset |
| | model-index: |
| | - name: functiongemma-270m-ecommerce-router |
| | results: |
| | - task: |
| | type: text-classification |
| | name: Agent Routing |
| | dataset: |
| | name: E-commerce Customer Support Routing |
| | type: scionoftech/ecommerce-agent-routing |
| | metrics: |
| | - type: accuracy |
| | value: 89.4 |
| | name: Routing Accuracy |
| | - type: f1 |
| | value: 89.0 |
| | name: Macro F1 Score |
| | --- |
| | |
| | # FunctionGemma 270M - E-Commerce Multi-Agent Router |
| |
|
| | Fine-tuned version of [google/functiongemma-270m-it](https://huggingface.co/google/functiongemma-270m-it) for intelligent routing of customer queries across 7 specialized agents in e-commerce customer support systems. |
| |
|
| | ## Model Description |
| |
|
| | This model demonstrates how FunctionGemma can be adapted beyond mobile actions for **multi-agent orchestration** in enterprise systems. It intelligently routes natural language customer queries to the appropriate specialized agent with **89.4% accuracy**. |
| |
|
| | **Key Achievement:** Replacing brittle rule-based routing (52-58% accuracy) with learned intelligence using only 1.47M trainable parameters (0.55% of the model). |
| |
|
| | ### Architecture |
| |
|
| | - **Base Model:** google/functiongemma-270m-it (270M parameters) |
| | - **Fine-tuning Method:** LoRA (Low-Rank Adaptation) |
| | - **Trainable Parameters:** 1,474,560 (0.55%) |
| | - **LoRA Rank:** 16 |
| | - **LoRA Alpha:** 32 |
| | - **Target Modules:** q_proj, k_proj, v_proj, o_proj |
| |
|
| | ### Training Details |
| |
|
| | - **Training Data:** 12,550 synthetic customer queries (balanced across 7 agents) |
| | - **Training Time:** 45 minutes on Google Colab T4 GPU |
| | - **Framework:** Hugging Face Transformers + PEFT + TRL |
| | - **Quantization:** 4-bit NF4 during training |
| | - **Optimizer:** paged_adamw_8bit |
| | - **Learning Rate:** 2e-4 |
| | - **Epochs:** 3 |
| | - **Batch Size:** 4 (effective 16 with gradient accumulation) |
| |
|
| | ## Intended Use |
| |
|
| | ### Primary Use Case |
| | **Multi-agent customer support routing** for e-commerce platforms: |
| | - Route queries to order management, product search, returns, payments, account, technical support agents |
| | - Maintain conversation context across multi-turn interactions |
| | - Enable intelligent task switching |
| |
|
| | ### Supported Agents |
| |
|
| | The model routes queries to 7 specialized agents: |
| |
|
| | 1. **Order Management** (`route_to_order_agent`) - Track orders, update delivery, cancel orders |
| | 2. **Product Search** (`route_to_search_agent`) - Search catalog, check availability, recommendations |
| | 3. **Product Details** (`route_to_details_agent`) - Specifications, reviews, comparisons |
| | 4. **Returns & Refunds** (`route_to_returns_agent`) - Initiate returns, process refunds, exchanges |
| | 5. **Account Management** (`route_to_account_agent`) - Update profile, manage addresses, security |
| | 6. **Payment Support** (`route_to_payment_agent`) - Resolve payment issues, update methods, billing |
| | 7. **Technical Support** (`route_to_technical_agent`) - Fix app/website issues, login problems |
| |
|
| | ### Out-of-Scope Use |
| |
|
| | - ❌ General-purpose chatbot (use base Gemma models instead) |
| | - ❌ Direct dialogue generation (this is a routing model) |
| | - ❌ More than 20 agents (context window limitations) |
| | - ❌ Non-customer-support domains without fine-tuning |
| |
|
| | ## Performance |
| |
|
| | ### Test Set Results |
| |
|
| | ``` |
| | Overall Accuracy: 89.40% (1,684/1,883 correct) |
| | |
| | Per-Agent Performance: |
| | order_management 92.3% (251/272) |
| | product_search 91.1% (257/282) |
| | product_details 94.7% (233/246) |
| | returns_refunds 88.2% (238/270) |
| | account_management 85.1% (229/269) |
| | payment_support 89.5% (241/269) |
| | technical_support 87.0% (234/269) |
| | ``` |
| |
|
| | ### Comparison to Baselines |
| |
|
| | | Approach | Accuracy | Latency | Memory | |
| | |----------|----------|---------|--------| |
| | | Keyword Matching | 52-58% | 5ms | Negligible | |
| | | Rule-based (100 rules) | 65-70% | 8ms | Negligible | |
| | | BERT Classifier (300M) | 82-85% | 45ms | 400 MB | |
| | | **This Model (LoRA)** | **89.4%** | **127ms** | **2.1 GB** | |
| | | GPT-4 API (zero-shot) | 85-90% | 2500ms | Cloud | |
| |
|
| | ### Latency Breakdown (T4 GPU) |
| |
|
| | - **Routing Decision:** 127ms average |
| | - **Agent Execution:** ~52ms average |
| | - **Total End-to-End:** ~179ms average |
| |
|
| | ## How to Use |
| |
|
| | ### Installation |
| |
|
| | ```bash |
| | pip install transformers peft torch accelerate bitsandbytes |
| | ``` |
| |
|
| | ### Quick Start |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | from peft import PeftModel |
| | import torch |
| | |
| | # Load base model |
| | base_model = AutoModelForCausalLM.from_pretrained( |
| | "google/functiongemma-270m-it", |
| | device_map="auto", |
| | torch_dtype=torch.bfloat16 |
| | ) |
| | |
| | # Load LoRA adapters |
| | model = PeftModel.from_pretrained( |
| | base_model, |
| | "scionoftech/functiongemma-270m-ecommerce-router" |
| | ) |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("google/functiongemma-270m-it") |
| | |
| | # Define available agents |
| | agent_declarations = """<start_function_declaration> |
| | route_to_order_agent(): Track, update, or cancel customer orders |
| | route_to_search_agent(): Search products, check availability |
| | route_to_details_agent(): Get product specifications and reviews |
| | route_to_returns_agent(): Handle returns, refunds, exchanges |
| | route_to_account_agent(): Manage user profile and settings |
| | route_to_payment_agent(): Resolve payment and billing issues |
| | route_to_technical_agent(): Fix app, website, login issues |
| | <end_function_declaration>""" |
| | |
| | # Route a query |
| | query = "Where is my order?" |
| | |
| | prompt = f"""<start_of_turn>user |
| | {agent_declarations} |
| | |
| | User query: {query}<end_of_turn> |
| | <start_of_turn>model |
| | """ |
| | |
| | inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| | |
| | with torch.no_grad(): |
| | outputs = model.generate( |
| | **inputs, |
| | max_new_tokens=30, |
| | do_sample=False, |
| | pad_token_id=tokenizer.eos_token_id |
| | ) |
| | |
| | response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=False) |
| | print(response) |
| | # Output: <function_call>route_to_order_agent</function_call> |
| | ``` |
| |
|
| | ### Production Deployment (4-bit Quantization) |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, BitsAndBytesConfig |
| | from peft import PeftModel |
| | |
| | # 4-bit quantization config |
| | quant_config = BitsAndBytesConfig( |
| | load_in_4bit=True, |
| | bnb_4bit_quant_type="nf4", |
| | bnb_4bit_compute_dtype=torch.bfloat16 |
| | ) |
| | |
| | # Load with quantization |
| | base_model = AutoModelForCausalLM.from_pretrained( |
| | "google/functiongemma-270m-it", |
| | quantization_config=quant_config, |
| | device_map="auto" |
| | ) |
| | |
| | model = PeftModel.from_pretrained( |
| | base_model, |
| | "scionoftech/functiongemma-270m-ecommerce-router" |
| | ) |
| | |
| | # Result: 180 MB model, 132ms latency, 89.1% accuracy |
| | ``` |
| |
|
| | ### Parsing Function Calls |
| |
|
| | ```python |
| | import re |
| | |
| | def extract_agent_function(response: str) -> str: |
| | """Extract function name from FunctionGemma output.""" |
| | match = re.search(r'<function_call>([a-zA-Z_]+)</function_call>', response) |
| | return match.group(1) if match else "unknown" |
| | |
| | # Usage |
| | agent = extract_agent_function(response) |
| | print(f"Route to: {agent}") |
| | # Output: Route to: route_to_order_agent |
| | ``` |
| |
|
| | ## Training Procedure |
| |
|
| | ### Dataset Preparation |
| |
|
| | Generated 12,550 synthetic examples with linguistic variations: |
| |
|
| | ```python |
| | # Example training format |
| | { |
| | "query": "Please track my package ASAP", |
| | "function": "route_to_order_agent", |
| | "agent": "order_management" |
| | } |
| | ``` |
| |
|
| | Variations included: |
| | - Polite forms: "Please", "Could you", "Can you" |
| | - Casual starters: "Hey", "Hi", "Um" |
| | - Urgency markers: "ASAP", "urgently", "immediately" |
| | - Edge cases and ambiguous queries |
| |
|
| | ### Training Configuration |
| |
|
| | ```python |
| | from transformers import TrainingArguments |
| | from trl import SFTTrainer |
| | from peft import LoraConfig |
| | |
| | # LoRA config |
| | lora_config = LoraConfig( |
| | r=16, |
| | lora_alpha=32, |
| | target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], |
| | lora_dropout=0.05, |
| | bias="none", |
| | task_type="CAUSAL_LM" |
| | ) |
| | |
| | # Training args |
| | training_args = TrainingArguments( |
| | output_dir="./functiongemma-ecommerce-router", |
| | num_train_epochs=3, |
| | per_device_train_batch_size=4, |
| | gradient_accumulation_steps=4, |
| | learning_rate=2e-4, |
| | lr_scheduler_type="cosine", |
| | warmup_ratio=0.1, |
| | weight_decay=0.01, |
| | bf16=True, |
| | optim="paged_adamw_8bit", |
| | logging_steps=20, |
| | eval_strategy="epoch", |
| | save_strategy="epoch" |
| | ) |
| | ``` |
| |
|
| | ### Training Results |
| |
|
| | - **Final Training Loss:** 0.0182 |
| | - **Final Validation Loss:** 0.0198 |
| | - **Training Time:** 45 minutes (T4 GPU) |
| | - **Peak Memory:** 11.2 GB |
| |
|
| | ## Limitations and Biases |
| |
|
| | ### Known Limitations |
| |
|
| | 1. **Ambiguous Queries:** 10.6% error rate concentrated in genuinely ambiguous queries |
| | - Example: "I need help" (could be any agent) |
| | - Mitigation: Implement confidence-based clarification (confidence < 0.7) |
| |
|
| | 2. **Context Dependency:** Requires conversation state management for multi-turn interactions |
| | - Solution: Use durable workflow orchestrators (Temporal, Cadence) |
| |
|
| | 3. **Agent Confusion:** Most common misclassifications: |
| | - Returns ↔ Order Management (12 cases) |
| | - Account ↔ Payment (8 cases) |
| | - Technical ↔ Product Details (6 cases) |
| |
|
| | 4. **Language:** Trained only on English queries |
| | - For multilingual support, fine-tune on translated datasets |
| |
|
| | ### Biases |
| |
|
| | - **Domain-Specific:** Trained exclusively on e-commerce customer support |
| | - **Synthetic Data:** Generated examples may not capture all real-world variations |
| | - **Agent Distribution:** Balanced training may not reflect real query distributions |
| |
|
| | ## Ethical Considerations |
| |
|
| | - **Misrouting Impact:** Incorrect routing may frustrate customers or delay issue resolution |
| | - **Recommendation:** Implement fallback to human agents for low-confidence predictions |
| | - **Privacy:** Model doesn't store user data; conversation state managed externally |
| | - **Fairness:** Ensure equal routing performance across user demographics |
| |
|
| | ## Citation |
| |
|
| | If you use this model in your research or production systems, please cite: |
| |
|
| | ```bibtex |
| | @misc{functiongemma-ecommerce-router, |
| | author = {Sai Kumar Yava}, |
| | title = {FunctionGemma 270M Fine-tuned for E-Commerce Multi-Agent Routing}, |
| | year = {2025}, |
| | publisher = {HuggingFace}, |
| | howpublished = {\url{https://huggingface.co/scionoftech/functiongemma-270m-ecommerce-router}}, |
| | } |
| | |
| | @article{functiongemma2025, |
| | title={FunctionGemma: Bringing bespoke function calling to the edge}, |
| | author={Google DeepMind}, |
| | year={2025}, |
| | url={https://blog.google/technology/developers/functiongemma/} |
| | } |
| | ``` |
| |
|
| | ## Acknowledgments |
| |
|
| | - Google DeepMind for FunctionGemma base model |
| | - Hugging Face for PEFT and Transformers libraries |
| | - The open-source AI community |
| |
|
| | ## License |
| |
|
| | This model inherits the Gemma license from the base model. See [Gemma Terms of Use](https://ai.google.dev/gemma/terms). |
| |
|
| | **Commercial Use:** Permitted under Gemma license terms. |
| |
|
| | ## Related Resources |
| |
|
| | - **Blog Post:** [Full implementation guide](https://medium.com/@saikumaryava/beyond-mobile-actions-exploring-functiongemma-for-intelligent-multi-agent-orchestration-242dc0273f93) |
| | - **Funcroute python package:** [funcroute](https://github.com/scionoftech/funcroute) |
| | - **Training Notebook:** [Google Colab](https://colab.research.google.com/github/scionoftech/functiongemma-finetuning-e-commerce/blob/main/FunctionGemma_fine_tuning.ipynb) |
| | - **GitHub Repository:** [Complete code](https://github.com/scionoftech/functiongemma-finetuning-e-commerce) |
| | - **Dataset:** [Training data](https://huggingface.co/datasets/scionoftech/functiongemma-e-commerce-dataset) |
| | - **Base Model:** [google/functiongemma-270m-it](https://huggingface.co/google/functiongemma-270m-it) |
| |
|
| | ## Updates |
| |
|
| | - **2025-12-25:** Initial release - 89.4% routing accuracy on e-commerce customer support |
| |
|
| | --- |
| |
|
| | **Questions?** Open an issue on [GitHub](https://github.com/scionoftech/functiongemma-finetuning-e-commerce/issues) |
