RASA not working with ollama - Rasa CALM - Rasa Community Forum

RASA not working with ollama

You have selected 0 posts.

post by Sanjukta.bs on Sep 12, 2024

This is my config file as using provider it is asking for open api key.

recipe: default.v1
language: en
pipeline:
- name: SingleStepLLMCommandGenerator
  llm:
    type: ollama
    model: llma3model
    base_url: http://localhost:11434

prompt_template: prompt_templates/time_aware_prompt.jinja2
  flow_retrieval:
    active: false

policies:
- name: FlowPolicy
  assistant_id: 20240911-121521-recursive-jersey

While it is generating response but it is taking a lot of time to generate that I keep getting this warning

(venv) D:\Sanjukta_rasa>rasa inspect
2024-09-12 13:30:49 INFO     rasa.tracing.config  - No endpoint for tracing type available in endpoints.yml,tracing will not be configured.
2024-09-12 13:31:01 INFO     root  - Connecting to channel 'rasa.core.channels.development_inspector.DevelopmentInspectInput' which was specified by the '--connector' argument. Any other channels will be ignored. To connect to all given channels, omit the '--connector' argument.
2024-09-12 13:31:02 INFO     root  - Starting Rasa server on http://0.0.0.0:5005
2024-09-12 13:31:23 INFO     rasa.core.processor  - Loading model models\20240912-132942-delicious-panel.tar.gz...
2024-09-12 13:31:23 WARNING  rasa.dialogue_understanding.generator.llm_based_command_generator  - [warning  ] Disabling flow retrieval can cause issues when there are a large number of flows to be included in the prompt. For more information see:
https://rasa.com/docs/rasa-pro/concepts/dialogue-understanding#how-the-llmcommandgenerator-works event_key=llm_based_command_generator.flow_retrieval.disabled
2024-09-12 13:31:25 INFO     root  - Rasa server is up and running.
[2024-09-12 13:31:26 +0530] [12816] [INFO] Starting worker [12816]
2024-09-12 13:31:26 INFO     sanic.server  - Starting worker [12816]
D:\Sanjukta_rasa\venv\lib\site-packages\langchain\llms\ollama.py:164: RuntimeWarning: coroutine 'AsyncCallbackManagerForLLMRun.on_llm_new_token' was never awaited
  run_manager.on_llm_new_token(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Can anyone explain how to work a way around of this? I have tried various configurations but none of them seems to work properly. Any responses are appreciated. Thanks.

post by Cabrell on Sep 23, 2024

Hi Did you find any solution for your problem? I’m facing the same problem

post by muzzammil on Sep 26, 2024

@Sanjukta.bs did you find a work around of this?

post by pkchoudhary1211 on Sep 27, 2024

Try enabling it again, as it can help with response times. Also, double-check that your API key.

post by Elliot94 on Sep 27, 2024

I get the same warning with Ollama, and the response times are so bad that its quicker to use GPT-4. But what API do you mean here exactly?