Kuldeep Singh Sidhu
singhsidhukuldeep
AI & ML interests
😃 TOP 3 on HuggingFace for posts 🤗 Seeking contributors for a completely open-source 🚀 Data Science platform! singhsidhukuldeep.github.io
Recent Activity
posted
an
update
about 1 hour ago
Exciting breakthrough in Text Embeddings: Introducing LENS (Lexicon-based EmbeddiNgS)!
A team of researchers from University of Amsterdam, University of Technology Sydney, and Tencent have developed a groundbreaking approach that outperforms dense embeddings on the Massive Text Embedding Benchmark (MTEB).
>> Key Technical Innovations:
- LENS consolidates vocabulary space through token embedding clustering, addressing the inherent redundancy in LLM tokenizers
- Implements bidirectional attention and innovative pooling strategies to unlock the full potential of LLMs
- Each dimension corresponds to token clusters instead of individual tokens, creating more coherent and compact embeddings
- Achieves competitive performance with just 4,000-8,000 dimensional embeddings, matching the size of dense counterparts
>> Under the Hood:
The framework applies KMeans clustering to token embeddings from the language modeling head, replacing original embeddings with cluster centroids. This reduces dimensionality while preserving semantic relationships.
>> Results:
- Outperforms dense embeddings on MTEB benchmark
- Achieves state-of-the-art performance when combined with dense embeddings on BEIR retrieval tasks
- Demonstrates superior performance across clustering, classification, and retrieval tasks
This work opens new possibilities for more efficient and interpretable text embeddings. The code will be available soon.
posted
an
update
2 days ago
Exciting breakthrough in Retrieval-Augmented Generation (RAG): Introducing MiniRAG - a revolutionary approach that makes RAG systems accessible for edge devices and resource-constrained environments.
Key innovations that set MiniRAG apart:
Semantic-aware Heterogeneous Graph Indexing
- Combines text chunks and named entities in a unified structure
- Reduces reliance on complex semantic understanding
- Creates rich semantic networks for precise information retrieval
Lightweight Topology-Enhanced Retrieval
- Leverages graph structures for efficient knowledge discovery
- Uses pattern matching and localized text processing
- Implements query-guided reasoning path discovery
Impressive Performance Metrics
- Achieves comparable results to LLM-based methods while using Small Language Models (SLMs)
- Requires only 25% of storage space compared to existing solutions
- Maintains robust performance with accuracy reduction ranging from just 0.8% to 20%
The researchers from Hong Kong University have also contributed a comprehensive benchmark dataset specifically designed for evaluating lightweight RAG systems under realistic on-device scenarios.
This breakthrough opens new possibilities for:
- Edge device AI applications
- Privacy-sensitive implementations
- Real-time processing systems
- Resource-constrained environments
The full implementation and datasets are available on GitHub: HKUDS/MiniRAG
posted
an
update
3 days ago
Exciting Research Alert: Multimodal Semantic Retrieval Revolutionizing E-commerce Product Search!
Just came across a fascinating paper from @amazon researchers that tackles a crucial challenge in e-commerce search - integrating both text and image data for better product discovery.
>> Key Innovations
The researchers developed two groundbreaking architectures:
- A 4-tower multimodal model combining BERT and CLIP for processing both text and images
- A streamlined 3-tower model that achieves comparable performance with reduced complexity
>> Technical Deep Dive
The system leverages dual-encoder architecture with some impressive components:
- Bi-encoder BERT model for processing text queries and product descriptions
- Visual transformers from CLIP for image processing
- Advanced fusion techniques including concatenation and MLP-based approaches
- Cosine similarity scoring for efficient large-scale retrieval
>> Real-world Impact
The results are remarkable:
- Up to 78.6% recall@100 for product retrieval
- Over 50% exact match precision
- Significant reduction in irrelevant results to just 11.9%
>> Industry Applications
This research has major implications for:
- E-commerce search optimization
- Visual product discovery
- Large-scale retrieval systems
- Cross-modal product recommendations
What's particularly impressive is how the system handles millions of products while maintaining computational efficiency through smart architectural choices.
This work represents a significant step forward in making online shopping more intuitive and accurate. The researchers from Amazon have demonstrated that combining visual and textual information can dramatically improve search relevance while maintaining scalability.
Organizations
singhsidhukuldeep's activity
Update Request
2
#2 opened 2 months ago
by
singhsidhukuldeep
The model can be started using vllm, but no dialogue is possible.
3
#2 opened 6 months ago
by
SongXiaoMao
Adding chat_template to tokenizer_config.json file
1
#3 opened 6 months ago
by
singhsidhukuldeep
Script request
3
#1 opened 6 months ago
by
singhsidhukuldeep
Requesting script
#1 opened 6 months ago
by
singhsidhukuldeep