Nicolay Rusnachenko

nicolay-r

AI & ML interests

Information Retrieval・Medical Multimodal NLP (🖼+📝) Research Fellow @BU_Research・software developer http://arekit.io・PhD in NLP

Recent Activity

posted an update 5 days ago

📢 Being inspired by effective LLM usage, delighted to share an approach that might boost your reasonging process 🧠 Sharing the demo for interactive launch of Chain-of-Thoght (CoT) schema in bash with the support of [optionally] predefined parameters as input files. The demo demonstrates application for author sentiment extraction towards object in text. This is a part of the most recent release of the bulk-chain 0.25.0. ⭐ https://github.com/nicolay-r/bulk-chain/releases/tag/0.25.1 How it works: it launches your CoT by asking missed parameters if necessary. For each item of the chain you receive input prompt and streamed output of your LLM. To settle onto certain parameters, you can pass them via `--src`: - TXT files (using filename as a parameter name) - JSON dictionaries for multiple 🤖 Model: https://huggingface.co./meta-llama/Llama-3.3-70B-Instruct 🌌 Other models: https://github.com/nicolay-r/nlp-thirdgate

replied to ychen's post 10 days ago

Here's some annoying keywords that 4o tends to use when responding to personal experiences with negative sentiments. Will be updated over time. `rough, tough, sound like, sounds like, frustrating, overwhelming`

replied to ychen's post 10 days ago

View all activity

Organizations

None yet

Posts 60

Post

608

📢 Being inspired by effective LLM usage, delighted to share an approach that might boost your reasonging process 🧠 Sharing the demo for interactive launch of Chain-of-Thoght (CoT) schema in bash with the support of [optionally] predefined parameters as input files. The demo demonstrates application for author sentiment extraction towards object in text.

This is a part of the most recent release of the bulk-chain 0.25.0.
⭐ https://github.com/nicolay-r/bulk-chain/releases/tag/0.25.1

How it works: it launches your CoT by asking missed parameters if necessary. For each item of the chain you receive input prompt and streamed output of your LLM.

To settle onto certain parameters, you can pass them via --src:
- TXT files (using filename as a parameter name)
- JSON dictionaries for multiple

🤖 Model: meta-llama/Llama-3.3-70B-Instruct
🌌 Other models: https://github.com/nicolay-r/nlp-thirdgate

Post

921

📢 If you're interesting in quick application of target sentiment analysis towards your data, you might be insterested in using fine-tuned FlanT5-xl version. Reason is a quick performance: I've added batching support for series of sentiment analysis models in this card:
nicolay-r/sentiment-analysis-advances-665ba391e0eba729021ea101

The provider implementation:
https://github.com/nicolay-r/nlp-thirdgate/blob/master/llm/transformers_flan_t5.py

📺 How to quick launch:
https://github.com/nicolay-r/bulk-chain/blob/master/test/test_provider_batching.py

Reason for using? experimenting in out-of domain, the noticed the performance of xl version similar to LLaMA-3-3b-instruct.

🔑 Key takeaways of adaptaiont:
- paddings and truncation strategies for batching mode:
- https://huggingface.co./docs/transformers/en/pad_truncation
- add_special_tokens=False causes a drastic changes in the result behaviour (FlanT5 models).
💥 Crashes on pad_token_id=50256 during generation proces.
🔻 use_bf16 mode performs 3 times slower on CPU.

🚀 Performance for BASE sized model:
nicolay-r/flan-t5-tsa-thor-base
17.2 it/s (prompt) and 5.22 it/s (3-step CoT) (CPU Core i5-1140G7)

There are other domain-oriented models could be launched via the same provider:
nicolay-r/flan-t5-emotion-cause-thor-base

Reference: https://github.com/huggingface/transformers/issues/26061

View all Posts