Post
1014
Designing Ecosystems of Intelligence from First Principles
(new major Friston paper)
https://arxiv.org/abs/2212.01354
(new major Friston paper)
https://arxiv.org/abs/2212.01354
Smol Community
community
AI & ML interests
The SmolTuners group is a community dedicated to the development of small-scale Large Language Models (LLMs) using consumer-grade GPUs.
Recent Activity
View all activity
SmolTuners's activity
KnutJaegersbergÂ
posted
an
update
6 days ago
KnutJaegersbergÂ
posted
an
update
8 days ago
Post
670
Mimicking Consciousness in LLMs: Ascending the Dimensions of Thought with Recurrent Processing
This blog post explores how **recurrent processing** can transform Large Language Models (LLMs) to mimic aspects of human thought by engaging in iterative feedback loops. Inspired by string theory, the post describes how LLMs can "ascend dimensions" of cognition, progressing through foundational cognitive loopsâsuch as basic cognition, executive functions, and meta-cognitionâbefore advancing into **world simulation**. In this stage, LLMs explore higher dimensions, perceiving non-linear time, simulating branching possibilities, and integrating multiple realities. The interaction between the **Generator** and **Reflective Compass** allows AI systems to refine their outputs iteratively, moving toward a **point attractor** where ideas become coherent and polished. While this process doesn't bestow true consciousness, it offers a compelling imitation of reflective and adaptive thinking, leading to smarter dialogue, enhanced creativity, and more robust problem-solving.
https://huggingface.co./blog/KnutJaegersberg/oscillatory-recurrence-for-llms
This blog post explores how **recurrent processing** can transform Large Language Models (LLMs) to mimic aspects of human thought by engaging in iterative feedback loops. Inspired by string theory, the post describes how LLMs can "ascend dimensions" of cognition, progressing through foundational cognitive loopsâsuch as basic cognition, executive functions, and meta-cognitionâbefore advancing into **world simulation**. In this stage, LLMs explore higher dimensions, perceiving non-linear time, simulating branching possibilities, and integrating multiple realities. The interaction between the **Generator** and **Reflective Compass** allows AI systems to refine their outputs iteratively, moving toward a **point attractor** where ideas become coherent and polished. While this process doesn't bestow true consciousness, it offers a compelling imitation of reflective and adaptive thinking, leading to smarter dialogue, enhanced creativity, and more robust problem-solving.
https://huggingface.co./blog/KnutJaegersberg/oscillatory-recurrence-for-llms
KnutJaegersbergÂ
posted
an
update
20 days ago
Post
2716
A Brief Survey of Associations Between Meta-Learning and General AI
The paper titled "A Brief Survey of Associations Between Meta-Learning and General AI" explores how meta-learning techniques can contribute to the development of Artificial General Intelligence (AGI). Here are the key points summarized:
1. General AI (AGI) and Meta-Learning:
- AGI aims to develop algorithms that can handle a wide variety of tasks, similar to human intelligence. Current AI systems excel at specific tasks but struggle with generalization to unseen tasks.
- Meta-learning or "learning to learn" improves model adaptation and generalization, allowing AI systems to tackle new tasks efficiently using prior experiences.
2. Neural Network Design in Meta-Learning:
- Techniques like Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks enable self-improvement and adaptability for deep models, supporting generalization across tasks.
- Highway networks and ResNet-style models use shortcuts for efficient backpropagation, allowing deeper models that can be used in meta-learning frameworks.
3. Coevolution:
- Coevolution involves the mutual evolution of multiple components, such as learners or task-solvers, to improve overall performance.
- Coevolution between learners enhances collaboration and competition within AI systems, while coevolution between tasks and solvers (e.g., POWERPLAY and AI-GA frameworks) pushes solvers to adapt to increasingly complex tasks.
4. Curiosity in Meta-Learning:
- Curiosity-based exploration encourages AI systems to discover new, diverse features of the environment, avoiding local optima.
- Curiosity-based objectives can be combined with performance-based objectives to ensure efficient exploration and adaptation in complex tasks.
5. Forgetting Mechanisms:
- Forgetting is crucial to avoid memory overload in AI systems
https://arxiv.org/abs/2101.04283
The paper titled "A Brief Survey of Associations Between Meta-Learning and General AI" explores how meta-learning techniques can contribute to the development of Artificial General Intelligence (AGI). Here are the key points summarized:
1. General AI (AGI) and Meta-Learning:
- AGI aims to develop algorithms that can handle a wide variety of tasks, similar to human intelligence. Current AI systems excel at specific tasks but struggle with generalization to unseen tasks.
- Meta-learning or "learning to learn" improves model adaptation and generalization, allowing AI systems to tackle new tasks efficiently using prior experiences.
2. Neural Network Design in Meta-Learning:
- Techniques like Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks enable self-improvement and adaptability for deep models, supporting generalization across tasks.
- Highway networks and ResNet-style models use shortcuts for efficient backpropagation, allowing deeper models that can be used in meta-learning frameworks.
3. Coevolution:
- Coevolution involves the mutual evolution of multiple components, such as learners or task-solvers, to improve overall performance.
- Coevolution between learners enhances collaboration and competition within AI systems, while coevolution between tasks and solvers (e.g., POWERPLAY and AI-GA frameworks) pushes solvers to adapt to increasingly complex tasks.
4. Curiosity in Meta-Learning:
- Curiosity-based exploration encourages AI systems to discover new, diverse features of the environment, avoiding local optima.
- Curiosity-based objectives can be combined with performance-based objectives to ensure efficient exploration and adaptation in complex tasks.
5. Forgetting Mechanisms:
- Forgetting is crucial to avoid memory overload in AI systems
https://arxiv.org/abs/2101.04283
KnutJaegersbergÂ
posted
an
update
21 days ago
Post
1654
Artificial general intelligence through recursive data compression and grounded reasoning: a position paper
This paper proposes a system to achieve AGI through general data compression and grounded reasoning.
General Data Compression involves creating a flexible algorithm that adapts to input data to simplify and compress it recursively, identifying simple, orthogonal features to avoid redundancy. The algorithm measures AGI progress by solving problems based on increasing complexity, and it expands its search space according to the data itself. Compression is applied not only to data but also to model parameters, and sequences are segmented based on compressibility.
Grounded Reasoning refers to forming representations with various granularities, crucial for commonsense reasoning and AGI. The system simulates the real world as its model, switching between representations and maximizing resourcefulness. Key ideas include the world as its own model for reasoning and actions aimed at maximizing entropy to test hypotheses.
The paper emphasizes simplicity, data-dependent bias, recursion, orthogonality, resourcefulness, and grounding in real-world contexts as fundamental principles in building an AGI system.
https://arxiv.org/abs/1506.04366
This paper proposes a system to achieve AGI through general data compression and grounded reasoning.
General Data Compression involves creating a flexible algorithm that adapts to input data to simplify and compress it recursively, identifying simple, orthogonal features to avoid redundancy. The algorithm measures AGI progress by solving problems based on increasing complexity, and it expands its search space according to the data itself. Compression is applied not only to data but also to model parameters, and sequences are segmented based on compressibility.
Grounded Reasoning refers to forming representations with various granularities, crucial for commonsense reasoning and AGI. The system simulates the real world as its model, switching between representations and maximizing resourcefulness. Key ideas include the world as its own model for reasoning and actions aimed at maximizing entropy to test hypotheses.
The paper emphasizes simplicity, data-dependent bias, recursion, orthogonality, resourcefulness, and grounding in real-world contexts as fundamental principles in building an AGI system.
https://arxiv.org/abs/1506.04366
- 1 reply
Post
2447
đ„đ Hey everyone! I'm excited to share my latest LLM release: Gilgamesh 72B, a model built on Qwen 2.5-72B Instruct. Gilgamesh was trained on a couple of my GammaCorpus datasets, specifically:
- rubenroy/GammaCorpus-CoT-Math-170k
- rubenroy/GammaCorpus-v2-5m
- rubenroy/GammaCorpus-Fact-QA-450k
I've submitted GGM 72B to the Open LLM Leaderboard for benchmarking, I'll send an update post once the results are in!
You can try it out and share your feedback, check out the model page and see what it can do:
đ rubenroy/Gilgamesh-72B
Would love to hear your thoughts!
- rubenroy/GammaCorpus-CoT-Math-170k
- rubenroy/GammaCorpus-v2-5m
- rubenroy/GammaCorpus-Fact-QA-450k
I've submitted GGM 72B to the Open LLM Leaderboard for benchmarking, I'll send an update post once the results are in!
You can try it out and share your feedback, check out the model page and see what it can do:
đ rubenroy/Gilgamesh-72B
Would love to hear your thoughts!
Post
2724
đ Fully released my newest models trained on my GammaCorpus dataset, Zurich 7B & 14B and Geneva 12B. Here is the model collections:
Zurich:
rubenroy/zurich-679b21284e207e2844bc025d
Geneva:
https://huggingface.co./collections/rubenroy/geneva-679e33a55d1576319b0d9cd4
If you would like to test them, feel free to visit their spaces:
rubenroy/Geneva-12B
rubenroy/Zurich-14B
rubenroy/Zurich-7B
Zurich:
rubenroy/zurich-679b21284e207e2844bc025d
Geneva:
https://huggingface.co./collections/rubenroy/geneva-679e33a55d1576319b0d9cd4
If you would like to test them, feel free to visit their spaces:
rubenroy/Geneva-12B
rubenroy/Zurich-14B
rubenroy/Zurich-7B
KnutJaegersbergÂ
posted
an
update
28 days ago
Post
1035
Anthropomorphic reasoning about neuromorphic AGI safety
Summary of "Anthropomorphic Reasoning About Neuromorphic AGI Safety"
This paper explores safety strategies for neuromorphic artificial general intelligence (AGI), defined as systems designed by reverse-engineering essential computations of the human brain. Key arguments and proposals include:
1. Anthropomorphic Reasoning Validity:
- Neuromorphic AGIâs design and assessment rely on human cognition models, making anthropomorphic reasoning (using human-like traits) critical for safety analysis. Comparisons to human behavior and neural mechanisms provide insights into AGI behavior and risks.
2. Countering Safety Criticisms:
- The authors challenge claims that neuromorphic AGI is inherently more dangerous than other AGI approaches. They argue all AGI systems face intractable verification challenges (e.g., real-world unpredictability, incomputable action validation). Neuromorphic AGI may even offer safety advantages by enabling comparisons to human cognitive processes.
3. Motivational Architecture:
- Basic drives (e.g., curiosity, social interaction) are essential for cognitive development and safety. These pre-conceptual, hardwired drives (analogous to human hunger or affiliation) shape learning and behavior. The orthogonality thesis (intelligence and goals as independent) is contested, as neuromorphic AGIâs drives likely intertwine with its cognitive architecture.
4. Safety Strategies:
- **Social Drives**: Embedding drives like caregiving, affiliation, and cooperation ensures AGI develops prosocial values through human interaction.
- **Bounded Reward Systems**: Human-like satiation mechanisms (e.g., diminishing rewards after fulfillment) prevent extreme behaviors (e.g., paperclip maximization).
- **Developmental Environment**: Exposure to diverse, positive human interactions and moral examples fosters
https://ccnlab.org/papers/JilkHerdReadEtAl17.pdf
Summary of "Anthropomorphic Reasoning About Neuromorphic AGI Safety"
This paper explores safety strategies for neuromorphic artificial general intelligence (AGI), defined as systems designed by reverse-engineering essential computations of the human brain. Key arguments and proposals include:
1. Anthropomorphic Reasoning Validity:
- Neuromorphic AGIâs design and assessment rely on human cognition models, making anthropomorphic reasoning (using human-like traits) critical for safety analysis. Comparisons to human behavior and neural mechanisms provide insights into AGI behavior and risks.
2. Countering Safety Criticisms:
- The authors challenge claims that neuromorphic AGI is inherently more dangerous than other AGI approaches. They argue all AGI systems face intractable verification challenges (e.g., real-world unpredictability, incomputable action validation). Neuromorphic AGI may even offer safety advantages by enabling comparisons to human cognitive processes.
3. Motivational Architecture:
- Basic drives (e.g., curiosity, social interaction) are essential for cognitive development and safety. These pre-conceptual, hardwired drives (analogous to human hunger or affiliation) shape learning and behavior. The orthogonality thesis (intelligence and goals as independent) is contested, as neuromorphic AGIâs drives likely intertwine with its cognitive architecture.
4. Safety Strategies:
- **Social Drives**: Embedding drives like caregiving, affiliation, and cooperation ensures AGI develops prosocial values through human interaction.
- **Bounded Reward Systems**: Human-like satiation mechanisms (e.g., diminishing rewards after fulfillment) prevent extreme behaviors (e.g., paperclip maximization).
- **Developmental Environment**: Exposure to diverse, positive human interactions and moral examples fosters
https://ccnlab.org/papers/JilkHerdReadEtAl17.pdf
KnutJaegersbergÂ
posted
an
update
about 1 month ago
Post
1884
Evolution and The Knightian Blindspot of Machine Learning
The paper discusses machine learning's limitations in addressing Knightian Uncertainty (KU), highlighting the fragility of models like reinforcement learning (RL) in unpredictable, open-world environments. KU refers to uncertainty that can't be quantified or predicted, a challenge that RL fails to handle due to its reliance on fixed data distributions and limited formalisms.
### Key Approaches:
1. **Artificial Life (ALife):** Simulating diverse, evolving systems to generate adaptability, mimicking biological evolution's robustness to unpredictable environments.
2. **Open-Endedness:** Creating AI systems capable of continuous innovation and adaptation, drawing inspiration from human creativity and scientific discovery.
3. **Revising RL Formalisms:** Modifying reinforcement learning (RL) models to handle dynamic, open-world environments by integrating more flexible assumptions and evolutionary strategies.
These approaches aim to address MLâs limitations in real-world uncertainty and move toward more adaptive, general intelligence.
https://arxiv.org/abs/2501.13075
The paper discusses machine learning's limitations in addressing Knightian Uncertainty (KU), highlighting the fragility of models like reinforcement learning (RL) in unpredictable, open-world environments. KU refers to uncertainty that can't be quantified or predicted, a challenge that RL fails to handle due to its reliance on fixed data distributions and limited formalisms.
### Key Approaches:
1. **Artificial Life (ALife):** Simulating diverse, evolving systems to generate adaptability, mimicking biological evolution's robustness to unpredictable environments.
2. **Open-Endedness:** Creating AI systems capable of continuous innovation and adaptation, drawing inspiration from human creativity and scientific discovery.
3. **Revising RL Formalisms:** Modifying reinforcement learning (RL) models to handle dynamic, open-world environments by integrating more flexible assumptions and evolutionary strategies.
These approaches aim to address MLâs limitations in real-world uncertainty and move toward more adaptive, general intelligence.
https://arxiv.org/abs/2501.13075
KnutJaegersbergÂ
posted
an
update
about 1 month ago
Post
2079
Artificial Kuramoto Oscillatory Neurons
Artificial Kuramoto Oscillatory Neurons (AKOrN) differ from traditional artificial neurons by oscillating, rather than just turning on or off. Each neuron is represented by a rotating vector on a sphere, influenced by its connections to other neurons. This behavior is based on the Kuramoto model, which describes how oscillators (like neurons) tend to synchronize, similar to pendulums swinging in unison.
Key points:
Oscillating Neurons: Each AKOrNâs rotation is influenced by its connections, and they try to synchronize or oppose each other.
Synchronization: When neurons synchronize, they "bind," allowing the network to represent complex concepts (e.g., "a blue square toy") by compressing information.
Updating Mechanism: Neurons update their rotations based on connected neurons, input stimuli, and their natural frequency, using a Kuramoto update formula.
Network Structure: AKOrNs can be used in various network layers, with iterative blocks combining Kuramoto layers and feature extraction modules.
Reasoning: This model can perform reasoning tasks, like solving Sudoku puzzles, by adjusting neuron interactions.
Advantages: AKOrNs offer robust feature binding, reasoning capabilities, resistance to adversarial data, and well-calibrated uncertainty estimation.
In summary, AKOrN's oscillatory neurons and synchronization mechanisms enable the network to learn, reason, and handle complex tasks like image classification and object discovery with enhanced robustness and flexibility.
yt
https://www.youtube.com/watch?v=i3fRf6fb9ZM
paper
https://arxiv.org/html/2410.13821v1
Artificial Kuramoto Oscillatory Neurons (AKOrN) differ from traditional artificial neurons by oscillating, rather than just turning on or off. Each neuron is represented by a rotating vector on a sphere, influenced by its connections to other neurons. This behavior is based on the Kuramoto model, which describes how oscillators (like neurons) tend to synchronize, similar to pendulums swinging in unison.
Key points:
Oscillating Neurons: Each AKOrNâs rotation is influenced by its connections, and they try to synchronize or oppose each other.
Synchronization: When neurons synchronize, they "bind," allowing the network to represent complex concepts (e.g., "a blue square toy") by compressing information.
Updating Mechanism: Neurons update their rotations based on connected neurons, input stimuli, and their natural frequency, using a Kuramoto update formula.
Network Structure: AKOrNs can be used in various network layers, with iterative blocks combining Kuramoto layers and feature extraction modules.
Reasoning: This model can perform reasoning tasks, like solving Sudoku puzzles, by adjusting neuron interactions.
Advantages: AKOrNs offer robust feature binding, reasoning capabilities, resistance to adversarial data, and well-calibrated uncertainty estimation.
In summary, AKOrN's oscillatory neurons and synchronization mechanisms enable the network to learn, reason, and handle complex tasks like image classification and object discovery with enhanced robustness and flexibility.
yt
https://www.youtube.com/watch?v=i3fRf6fb9ZM
paper
https://arxiv.org/html/2410.13821v1
- 2 replies
KnutJaegersbergÂ
posted
an
update
about 1 month ago
Post
1594
DeepSeek R1 on how to build conscious AGI
https://huggingface.co./blog/KnutJaegersberg/deepseek-r1-on-conscious-agi
https://huggingface.co./blog/KnutJaegersberg/deepseek-r1-on-conscious-agi
-
- 3 replies
KnutJaegersbergÂ
posted
an
update
about 1 month ago
Post
497
Yet another blog post about general intelligence
https://huggingface.co./blog/KnutJaegersberg/general-intelligence
https://huggingface.co./blog/KnutJaegersberg/general-intelligence
KnutJaegersbergÂ
posted
an
update
about 1 month ago
Post
1778
Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGI
It's an interesting paper that argues "new approaches are required that can reliably solve a wide variety of problems without existing skills."
"It is therefore hoped that the benchmark outlined in this article contributes to further exploration of this direction of research and incentivises the development of new AGI approaches that focus on intelligence rather than skills."
https://arxiv.org/abs/2501.07458
It's an interesting paper that argues "new approaches are required that can reliably solve a wide variety of problems without existing skills."
"It is therefore hoped that the benchmark outlined in this article contributes to further exploration of this direction of research and incentivises the development of new AGI approaches that focus on intelligence rather than skills."
https://arxiv.org/abs/2501.07458
s3nhÂ
Gh organization
8
#3 opened 2 months ago
by
s3nh
Gh organization
8
#3 opened 2 months ago
by
s3nh
KnutJaegersbergÂ
posted
an
update
about 2 months ago
Gh organization
8
#3 opened 2 months ago
by
s3nh
Gh organization
8
#3 opened 2 months ago
by
s3nh
s3nhÂ
updated
a
Space
2 months ago
Gh organization
8
#3 opened 2 months ago
by
s3nh
s3nhÂ
Optimizers
#2 opened 2 months ago
by
s3nh