cleveres-ai

KO | EN

gitlite — search

#javascript #python #ai #typescript #rust #deep-learning #windows #cli #kubernetes #vue #pytorch #ios #openai #open-source

(★ 17)

A smart knowledge base about artificial intelligence.

cleveres-ai 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# cleveres-ai
A smart knowledge base about artificial intelligence.

## Frontier Models
*   [Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model](models/mage-vl.md) - *An efficient codec-native streaming foundation model for real-time multimodal understanding and interaction that selectively encodes dynamic regions using motion vectors.*
*   [Kimi K3: Open Frontier Intelligence](models/kimi-k3.md) - *A 2.8T parameter Mixture-of-Experts model with native vision and a 1-million-token context, achieving frontier-level performance through Stable LatentMoE and Kimi Delta Attention.*
*   [Qwen-Music Technical Report](models/qwen-music.md) - *A powerful music generation model capable of producing highly musical and high-fidelity songs with complete vocal singing using a melody-token-based chain-of-thought.*
*   [Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation](models/boogu-image-0-1.md) - *We introduce Boogu-Image-0.1, an open-source unified multimodal understanding and generation model family, comprising Base, Turbo, Edit, and Edit-Turbo variants. It delivers competitive performance in high-quality text-to-image generation, fast inference, instruction-based editing, and bilingual (Chinese-English) text rendering. Closed-source multimodal systems like Nano-Banana-Pro and GPT-Image-2 achieve strong performance through system-level integration rather than a single model, yet their internal practices remain largely undisclosed. In this work, we demonstrate that targeted improvements in model understanding, data quality, and training pipelines, coupled with agentic inference-time scaling, can substantially enhance generation and editing performance even under highly constrained compute budgets. Comprehensive evaluations show that Boogu-Image-0.1 consistently matches or surpasses other open-source models across standard benchmarks, and achieves results approaching leading closed-source systems. Notably, this is accomplished with only 208.62 million unique images. The base model's theoretical training cost is only approximately \$400K. We share practical discussions that we believe are valuable to the broader research community, and release weights, code, and recipes under Apache 2.0 to advance the open ecosystem for unified multimodal understanding and generation. Our code is available here: https://github.com/Boogu-Project/Boogu-Image.*
*   [Vidu S1: A Real-Time Interactive Video Generation Model](models/vidu-s1.md) - *A real-time interactive video generation model supporting voice control of digital characters and infinite-length generation without blurring.*
*   [Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity](models/seed2-0.md) - *A model series targeting real-world complexity and long-tail knowledge by substantially improving instruction following and reliability on intricate, long-horizon tasks.*
*   [Sumi: Open Uniform Diffusion Language Model from Scratch](models/sumi-uniform-diffusion.md) - *A fully open 7B uniform diffusion language model pretrained from scratch.*
*   [VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion](papers/video-mla.md) - *Applies Multi-Head Latent Attention to video diffusion, reducing per-token KV memory by 92.7% at every cached layer.*
*   [Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning](papers/equilibrium-reasoners.md) - *A framework for scalable reasoning using latent dynamical systems and attractors, unrolling up to 40,000 layers.*
*   [OpenSeeker-v2: SOTA Search Agent via Pure SFT](models/openseeker-v2.md) - *An open-source 30B search agent that achieves state-of-the-art performance using only Supervised Fine-Tuning on 10.6k high-difficulty trajectories, proving heavy RL pipelines are not strictly required.*
*   [Qwen3-Omni: Native Omni-Modal Foundation Model](models/qwen3-omni.md) - *Alibaba's end-to-end multilingual model processing text, image, audio, and video with real-time streaming.*
*   [Mistral Large 3: The 675B Multimodal MoE](models/mistral-large-3.md) - *Mistral AI's frontier model optimized for enterprise deployment.*
*   [Talkie-1930: The 13B Vintage Language Model](models/talkie-1930.md) - *A 13B parameter open-weight language model trained exclusively on pre-1931 English text to study generalization and LLM identity.*
*   [DeepSeek-V4: The Next Generation](models/deepseek-v4.md) - *The next generation of DeepSeek models, featuring Pro and Flash variants.*
*   [EXAONE 4.5: LG's Open-Weight Vision Language Model](models/exaone-4-5.md) - *A multimodal model extending context up to 256K tokens for document-centric enterprise applications.*
*   [ECHO: Efficient Chest X-ray Report Generation](models/echo-chest-xray.md) - *A diffusion-based VLM reducing latency via one-step-per-block inference.*
*   [GLM-5.1: The 754B Agentic MoE Model](models/glm-5-1.md) - *Z.ai's open-source MoE model engineered for autonomous work, beating Opus 4.6 on SWE-Bench Pro.*
*   [OpenVLThinkerV2: Generalist Multimodal Reasoning Model](models/openvlthinkerv2.md) - *A general-purpose multimodal model utilizing Gaussian GRPO (G²RPO) to balance perception and multi-step reasoning.*
*   [Gemma 4: Google's Most Capable Open Models](models/gemma-4.md) - *Google's new family of open models (E2B, E4B, 26B MoE, 31B Dense) for advanced reasoning.*
*   [Trinity Large Thinking: Arcee's 400B Open-Source Model](models/trinity-large-thinking.md) - *A massive 400B-parameter open-source language model providing a Western alternative to Chinese models.*
*   [VibeVoice: Microsoft's Frontier Long Conversational TTS](models/vibevoice.md) - *An open-source framework for generating expressive, long-form multi-speaker audio.*
*   [Gemini 2.5 Pro Experimental: Google's Thinking Model](models/gemini-2-5-pro.md) - *Google's state-of-the-art thinking model scoring 63.8% on SWE-Bench Verified.*
*   [Voxtral 4B TTS: Mistral AI's Multilingual Text-to-Speech](models/voxtral-4b-tts.md) - *Mistral AI's model for generating realistic speech in 9 languages.*
*   [Granite-Docling-258M: Ultra-Compact VLM for Document Conversion](models/granite-docling-258m.md) - *IBM's 258M parameter model for accurate and efficient document conversion.*
*   [QwQ-32B: The 32B Parameter Open-Source Reasoning Model](models/qwq-32b.md) - *Alibaba's open-source 32B model built for advanced reasoning and coding.*
*   [Gemma 3: Google's Open-Weights Multimodal Model](models/gemma-3.md) - *A family of lightweight models from Google DeepMind capable of multimodal understanding.*
*   [ZeroClaw: The Ultra-Lightweight Rust AI Agent Framework](models/zeroclaw.md) - *A blazing-fast, <5MB memory AI agent framework built entirely in Rust.*
*   [Nano Banana 2: Gemini 3.1 Flash Image](models/nano-banana-2.md) - *Google's latest fast image generation model.*
*   [Nemotron 3 Super: The Hybrid Mamba-MoE](models/nemotron-3-super.md) - *NVIDIA's 120B model (12B active) designed for agentic reasoning.*
*   [MiniMax M2.7: SOTA Open Model](models/minimax-m2-7.md) - *MiniMax's SOTA open model matching GLM-5 at 1/3 the cost.*
*   [Xiaomi MiMo-V2-Pro: The API-Only Reasoning Engine](models/xiaomi-mimo-v2-pro.md) - *Xiaomi's API-only reasoning entrant with a 1M context window.*
*   [Gemini 3.1 Flash-Lite: The Cost-Efficient High-Volume Model](models/gemini-3-1-flash-lite.md) - *Google's fastest and most cost-efficient Gemini 3 series model, optimized for high-volume, low-latency tasks.*
*   [Gemini 3.1 Pro: The Expanded Context Engine](models/gemini-3-1-pro.md) - *Google's frontier reasoning model with a 1M token context window, natively multimodal.*
*   [Claude Opus 4.6: The Long-Horizon Reasoning Engine](models/claude-opus-4-6.md) - *Anthropic's newly released model with a 1M token context window, engineered for long-horizon reasoning and autonomous agentic work.*
*   [GPT-5.4 mini and nano: High-Volume Efficiency](models/gpt-5-4-mini-and-nano.md) - *OpenAI's latest highly efficient models optimized for speed, cost, and high-volume workloads.*
*   [Xiaomi MiMo: The "Hunter Alpha" Stealth Model](models/hunter-alpha-mimo.md) - *Xiaomi's stealth model that surfaced anonymously and sparked widespread speculation.*
*   [Mistral Small 4: The Unified MoE Model](models/mistral-small-4.md) - *Mistral AI's 119B hybrid MoE model combining reasoning, multimodal, and agentic capabilities.*
*   [Grok 3: The Age of Reasoning Agents](models/grok-3.md) - *xAI's newest frontier model focusing on test-time compute and advanced mathematical reasoning.*
*   [GLM-5: The 744B Frontier MoE Model](models/glm-5.md) - *Zhipu AI's flagship open-weights model featuring DeepSeek Sparse Attention and asynchronous RL for complex agentic tasks.*
*   [Claude 3.7 Sonnet: The Hybrid Reasoning Engine](models/claude-3-7-sonnet.md) - *Anthropic's hybrid model that toggles between instant responses and extended thinking.*
*   [Wan 2.1: The Open-Source Video Generator](models/wan-2-1.md) - *Alibaba's 14B parameter video generation model with advanced motion control.*
*   [Qwen 2.5-VL: The Vision-Language Flagship](models/qwen-2-5-vl.md) - *A multimodal model for native dynamic resolution, long-video comprehension, and visual agent tasks.*
*   [DeepSeek-R1: The Open-Source Reasoning Champion](models/deepseek-r1.md) - *The first open-weights model to rival o1 using Pure RL.*
*   [DeepSeek-V3: The Open-Source Titan Behind R1](models/deepseek-v3.md) - *The 671B MoE base model for R1 with innovative MLA and DeepSeekMoE.*
*   [DeepSeek-V3.2: The Sparse Attention Evolution](models/deepseek-v3-2.md) - *The 685B model that introduces DeepSeek Sparse Attention (DSA) for efficient long-context processing.*
*   [OpenAI o3-mini: High-Speed Reasoning for the Masses](models/o3-mini.md) - *A fast, cost-efficient reasoning model tailored for STEM and coding tasks.*
*   [OpenManus: The Open-Source Framework for General AI Agents](models/openmanus.md) - *An open-source alternative to Manus for creating and customizing advanced agentic workflows without an invite code.*

## Frontier Research & Papers
*   [Persistent Recursive Worlds Enable Autonomous Software Evolution](papers/evox-genesis-persistent-worlds.md) - *Introduces Genesis, a framework representing software as a persistent recursive world where finite-lived agents propose changes, shifting the organization of long-horizon development from persistent agents to persistent projects.*
*   [Not Worth Another Token: Marginal Value Estimation for Efficient Deep Research Agents](papers/not-worth-another-token.md) - *An evaluation of pruning strategies across the pipeline of long-horizon research agents, showing that early heuristic pruning can reduce token usage by up to 73% with minimal quality degradation.*
*   [OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching](papers/oasis-kv.md) - *A memory-centric LLM inference system that alleviates HBM capacity pressure by decoupling full KV-cache storage from HBM during decoding, using lookahead tokens from speculative decoding to prefetch relevant KV blocks.*
*   [Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution](papers/ouroboros.md) - *A self-developing coding agent whose tools, prompts, and core implementation improve through reviewed commits via recursive free evolution and experience-driven core evolution.*
*   [When Privileged Guidance Misaligns: State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents](papers/smrc-sd.md) - *Introduces SMRC-SD, an on-policy distillation framework for multi-turn agents that prevents state-reference mismatch by only distilling at matched states and conditioning supervision on the current execution context.*
*   [FactorJEPA: Factorizing Monolithic Futures into Layout-Agent-Interaction Channels for Crowded and Chaotic Global South Urban Worlds](papers/factorjepa-denseworld.md) - *Introduces FactorJEPA, a composite world model, and the DENSEWORLD-115k dataset to effectively handle the high density, agent heterogeneity, and persistent occlusion typical of Global South urban environments.*
*   [Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills](papers/weights-vs-skills-robot-learning.md) - *A survey categorizing robot learning into static weight-based policies (VLA models) and dynamic skill-based code synthesis, highlighting the limitations of current commercial skill marketplaces.*
*   [KVAE: Family of Tokenizers for Multimodal Generative Models](papers/kvae-family-tokenizers.md) - *A comprehensive series of continuous and discrete tokenizers (KVAE-Audio, KVAE-3D, KVAE-2D) designed specifically to match or surpass frontier open-source alternatives for text-conditioned generation.*
*   [Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay](papers/activity-frames-deterministic-memory.md) - *A zero-model compiler that parses continuous local screen capture into structured, memory-efficient deterministic frames for autonomous agents, avoiding expensive frontier LLM calls.*
*   [The Bitter Lesson of Tool Calling](papers/bitter-lesson-tool-calling.md) - *An empirical study demonstrating that programmatic tool calling (PTC), where tools are exposed as Python stubs and invoked via code, systematically outperforms rigid native JSON tool calling across language models.*
*   [Learning When to Trust via Selective Context Preference Optimization](papers/selective-context-preference-optimization.md) - *A benchmark (MIST) and DPO-based optimization approach (SCOPE) to train language models on selective trust, ensuring they robustly ignore misleading context without unconditionally rejecting correct external signals.*
*   [Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance](papers/rstg-adaptive-teacher-guidance.md) - *An adaptive teacher guidance framework (RSTG) that stabilizes GRPO by selectively applying on-policy distillation only to negative, zero-variance prompts and high-divergence tokens.*
*   [Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent](papers/video-deepresearch.md) - *A multimodal agent framework extending static image reasoning to continuous video streams by decoupling perception and exploration, using GRPO to break imitation learning limits and prevent modality bias.*
*   [OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models](papers/omnipack-token-compression.md) - *A training-free token compression framework for omni-modal LLMs that coordinates structural redundancy removal before the LLM with task-relevant semantic refinement inside it, achieving high performance preservation while drastically reducing FLOPs.*
*   [SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation](papers/skt-skill-use-training.md) - *A verified data synthesis pipeline that constructs skill-grounded tasks and executable trajectories to improve language-model agents' ability to identify, apply, and coordinate skills.*
*   [Progressive Agent Skill Generation via Reinforcement Learning](papers/skill-alpha-progressive-agent-skill-generation.md) - *A reinforcement learning method (Skill-$\alpha$) for progressively generating high-quality agent skills formulated as a sequential editing process evaluated by a novel rollback reward.*
*   [Enhancing Rubric-based RL via Self-Distillation](papers/enhancing-rubric-based-rl-self-distillation.md) - *A framework addressing Unexplored and Suppressed Criteria in rubric-based RL by employing on-policy self-distillation to explicitly teach missing behaviors and preserve suppressed token patterns without inference mismatch.*
*   [Meshy T2: Fast Native Mesh Generation with Flow Matching](papers/meshy-t2-fast-native-mesh-generation.md) - *A fast native mesh generation framework using a vertex-set mesh VAE and flow matching to generate continuous 3D assets interactively.*
*   [β-OPSD: Deriving with Policy Optimization, Training with Self-Distillation](papers/b-opsd.md) - *A generalized on-policy self-distillation method that introduces a controllable regularization parameter β to trade off proximity to a reference policy against privileged teacher guidance, improving optimization stability.*
*   [Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation](papers/explorative-modeling.md) - *A new paradigm that factors the training loop by exploring multiple candidate matches between generations and data, introducing a third pretraining axis for generative models.*
*   [Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes](papers/revisiting-lossy-verification.md) - *An analysis of lossy verification schemes in speculative decoding, revealing that truncation-based methods can degrade quality compared to the sampling baseline due to distributional distortion.*
*   [SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution](papers/skillrise.md) - *A unified RL framework for learning skills across tasks by alternating between task solving and curating an evolving skill document.*
*   [CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization](papers/cort.md) - *A token-level credit weighting method for rubric-conditioned GRPO that uses counterfactual replay to compute likelihood contrasts without an auxiliary scoring model.*
*   [From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search](papers/multi-agent-protocol-distillation.md) - *A joint distillation and RL framework bridging the distribution gap in agentic search by using a structured, style-normalized protocol to transfer dense reasoning signals from proprietary teachers to open-source students.*
*   [Agentic Context Management: Solving Agent Memory and Cost by Treating Them as Lifecycle and Architecture Problems](papers/agentic-context-management.md) - *A framework that treats agent context management as a comprehensive lifecycle problem (architecting, ingesting, scoping, anticipating, compacting) rather than just storage-and-retrieval, reducing token costs while preserving fidelity.*
*   [OpenForgeRL: Train Harness-native Agents in Any Environment](papers/openforgerl.md) - *An open-source framework for training harness-based agents end-to-end in diverse environments by decoupling training and inference.*
*   [Dataset Distillation by Influence Matching](papers/dataset-distillation-influence-matching.md) - *A dataset distillation method that aligns the final outcome of training using a fully differentiable, sample-level influence estimator.*
*   [Beyond Euclidean Clipping: Overcoming Exploration Collapse in LLM RL via Riemannian Isometric Policy Optimization](papers/ripo-riemannian-isometric-policy-optimization.md) - *A method that corrects the geometric flaw in PPO-Clip by enforcing isometric policy updates on the Riemannian manifold, effectively balancing exploration and exploitation.*
*   [AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents](papers/agentdebugx.md) - *An open-source debugging framework providing failure observability, root-cause attribution, and recovery for LLM agents.*
*   [Distilled Reinforcement Learning for LLM Post-training](papers/distilled-reinforcement-learning.md) - *A hybrid approach combining teacher supervision with RL to transfer new knowledge without the pitfalls of standard on-policy distillation.*
*   [SWE-Pruner Pro: The Coder LLM Already Knows What to Prune](papers/swe-pruner-pro.md) - *An in-agent context pruning method that uses a small head on internal representations to discard irrelevant tool outputs, saving tokens and preserving task quality.*
*   [Rethinking the Evaluation of Harness Evolution for Agents](papers/rethinking-harness-evolution.md) - *An evaluation of automatic harness evolution showing that when compared fairly against simple test-time scaling baselines under matched budgets, it fails to consistently outperform and struggles to generalize to held-out tasks.*
*   [Token Time Continuous Diffusion for Language Modeling](papers/token-time-continuous-diffusion.md) - *In this paper we introduce token time continuous diffusion (TTCD), a new diffusion language model which (a) operates in continuous space, deterministically mapping Gaussian noise to a final token canvas with no further sampling, and crucially (b) incorporates a new notion of per-token times, with some tokens proceeding from noise to token at a faster rate than others. Continuous space modeling helps TTCD avoid the parallel sampling of multiple tokens, which is a key source of inaccuracy at high speedups for models that iterate purely in discrete space. The notion of per-token times helps TTCD to better model conditional generation, allows for more sure tokens to proceed at a faster rate, and allows for differentiated inter-token influences during refinement. TTCD outperforms discrete models at high speedups. We train a 160M parameter TTCD model on OpenWebText, and then self-distill it; we find that at high speedups we are comparable in unconditional generation quality, and outperform in conditional generation, several existing models of similar size trained, on the same data, and self-distilled. We achieve similar gains in Sudoku solving as well.*
*   [LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget](papers/longstraw-long-context-rl.md) - *A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially important for AI agents, whose observations, tool outputs, documents, and prior decisions accumulate over long trajectories. LongStraw is an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, instantiated with Group Relative Policy Optimization (GRPO). It evaluates the shared prompt without autograd, retains only model-specific state needed by later tokens, and replays short response branches one at a time, reducing the live training graph at the cost of additional replay time. We implement it for the hybrid recurrent and full-attention Qwen3.6-27B and the compressed-attention mixture-of-experts GLM-5.2. On eight H20 GPUs, LongStraw completes grouped Qwen scoring and response backward at 2.1M positions for groups of 2 and 8; increasing the group size adds only 0.21 GB of peak allocated memory, while a separate stress test reaches 4.46M positions. On 32 H20 GPUs, we validate the end-to-end LongStraw execution path for a 2.1M-token prompt across all 78 layers of GLM-5.2. These experiments establish execution capacity rather than complete training correctness because the captured prompt state is detached and some distributed forward and gradient composition paths remain incomplete.*
*   [Spectral Rewiring for Exploration, Purification, and Model Merging](papers/spectral-rewiring-exploration-purification.md) - *Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging. We show that the reasoning-effective component of these updates is largely concentrated in the base model's spectral space, motivating Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains this spectral core while removing orthogonal components. SAR therefore preserves reasoning gains and filters residual update directions that suppress performance or amplify cross-domain interference. Across several model  families and scales, SAR extracts compact reasoning cores using as little as approximately 0.58% of total parameters: it preserves over 99% of post-training performance and improves high-k exploration in mathematical reasoning, and generalizes to agentic coding by improving six of seven open benchmarks on an in-house model. SAR also purifies mixed-domain training updates by releasing suppressed coding capability while maintaining math reasoning and instruction following. It further enables model merging across experts, yielding cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts. Overall, SAR shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.*
*   [DeepLoop: Depth Scaling for Looped Transformers](papers/deeploop-depth-scaling-looped-transformers.md) - *Looped Transformers scale sequential computation by applying a compact stack of physical blocks for multiple rounds, increasing unrolled depth without increasing stored parameters. This reuse changes the residual-scaling problem: in an untied Transformer, each residual branch receives and applies its own parameter update, whereas in a looped Transformer one shared update aggregates gradients from repeated visits and is read back by those same visits in the next linearized forward pass. We formalize this tied-depth effect through a first-order perturbation bound controlled by a visit-alignment coefficient κ_R. The bound recovers the DeepNorm exponent when visits decorrelate, but in the conservative aligned regime it requires the exponent to increase from 1/4 to 1/2 as loop count grows at fixed physical depth. The resulting method, DeepLoop, keeps the Post-LN DeepNorm architecture and sets α=(2N)^{1/2} and β=(8N)^{-1/2} for unrolled depth N. On GPT-style looped language models at GPT-2 small and GPT-2 medium scale, DeepLoop is neutral when no physical block is revisited and improves validation loss and downstream accuracy once recurrent depth is activated. These results show that stable recurrent depth requires residual scaling rules that account for parameter visits, not only nominal layer count.*
*   [Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model](papers/xiaomi-robotics-u0.md) - *Recent foundation image and video generation models offer strong generalization and controllability, but their direct application to embodied scenarios is limited by requirements for multi-view consistency, geometric coherence, and robot embodiment constraints. Existing methods typically adapt foundation models with limited robot data, often sacrificing visual knowledge acquired during large-scale pre-training. We present Xiaomi-Robotics-U0, a 38-billion-parameter multimodal autoregressive model for unified embodied synthesis. It treats embodied generation as an extension of foundation image and video generation and jointly optimizes text-to-image generation, image editing, embodied scene generation, embodied transfer, and embodied video generation. This unified framework preserves the generalization of the pre-trained world foundation model while adapting it to embodied settings. Xiaomi-Robotics-U0 is the first model to support high-quality multi-view scene generation across multiple robot embodiments and to introduce structured, controllable embodied transfer for fine-grained editing while preserving multi-view consistency and interaction dynamics. It achieves state-of-the-art results on single-step and sequential generation tasks, outperforming GPT-Image-2.0 in human evaluations of embodied scene generation and transfer, ranking first on World Arena for embodied video generation, and improving the out-of-distribution success rate of pi_0.5 from 36.9% to 63.2% on challenging real-world manipulation tasks. These results show that foundation world models can serve both as embodied world models and scalable data engines for embodied intelligence. Code and checkpoints are available at https://robotics.xiaomi.com/xiaomi-robotics-u0.html.*
*   [Multi-Agent LLMs Fail to Explore Each Other](papers/multi-agent-llms-fail-to-explore-each-other.md) - *Exploration is essential for reliable autonomy in multi-agent systems, yet it remains unclear whether large language model (LLM) agents can explore effectively when interacting with one another. We show that modern LLM agents fail to do so, often exhibiting myopic and polarized interaction patterns that lead to suboptimal coordination and increased regret. We formalize this challenge as the Multi-Agent Exploration problem, modeling it as a partially observable stochastic game (POSG) problem in which agents must probe peers to infer their capabilities and identify effective interaction strategies. To address this, we introduce Multi- Agent Contextual Exploration (MACE), a lightweight framework that explicitly promotes exploration through structured peer selection. Across both contextual and parametric diversity settings, MACE substantially improves exploration behavior and downstream task performance. We further show theoretically that the value of exploration increases with agent diversity. Overall, our results highlight a fundamental limitation of current LLM agents and underscore the importance of explicitly guided exploration for reliable multi-agent autonomy. Code will be released in https://github.com/deeplearning-wisc/mace*
*   [Weak-to-Strong Generalization via Direct On-Policy Distillation](papers/direct-on-policy-distillation.md) - *A weak-to-strong training method that avoids expensive target-model rollouts by running RL on a smaller model and directly transferring the induced policy shift as a dense implicit reward.*
*   [Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning](papers/knowing-using-gap.md) - *Formalizes the Knowing-Using Gap and introduces self-patching to diagnose why memorized knowledge often fails to route to computation-effective layers during reasoning tasks.*
*   [KronQ: LLM Quantization via Kronecker-Factored Hessian](papers/kronq-llm-quantization.md) - *A post-training quantization framework that introduces the gradient covariance into the quantization pipeline, achieving 7.93 perplexity in the case of 2-bit weight-only quantization on LLaMA-3-70B.*
*   [Self-Guided Test-Time Training for Long-Context LLMs](papers/self-guided-test-time-training.md) - *A method that improves long-context processing by having the model identify and adapt on relevant evidence spans via test-time training, avoiding noise from randomly sampled spans.*
* [On the Limitations of Non-GPU AI Accelerators for Large-Model Inference: A Field Study of MoE and Multimodal Serving on Huawei Ascend](papers/limitations-of-nongpu-ai-accelerators.md) - *A field study documenting the real engineering costs, system limitations, and workarounds needed to deploy demanding workloads (like W8A8 MoE and multimodal serving) on non-GPU AI accelerators like Huawei Ascend.*
* [CineMobile: On-Device Image-to-Video Diffusion for Cinematic Camera Motion Generation](papers/cinemobile-on-device-image-to-video-diffusion.md) - *An optimized on-device diffusion framework that achieves a 40x speedup for cinematic image-to-video generation on mobile platforms with a compressed sub-1GB footprint.*
* [Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents](papers/proactive-memory-agent.md) - *A proactive memory agent running alongside an action agent that selectively injects memory-grounded reminders to mitigate behavioral state decay in long-horizon tasks.*
* [Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing](papers/linear-attention-architectures-routing.md) - *A comparative study of recurrent linear-attention architectures, including DeltaNet and Gated DeltaNet-2, and an evaluation of cross-layer routing mechanisms.*
* [Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models](papers/flash-bon.md) - *Generates a large pool of inexpensive draft candidates by combining three complementary acceleration knobs for efficient inference-time scaling in diffusion models.*
* [Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training](papers/is-one-layer-enough-rl-training.md) - *Training a single transformer layer can recover most of the gains achieved by full-parameter RL training.*
* [SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Review](papers/swe-review-agentic-code-review.md) - *A framework for closing the one-shot PR generation loop with an agentic code review system.*
*   [MentalThink: Shaping Thoughts in Mental SVG World](papers/mentalthink.md) - *A visual-symbolic reasoning paradigm that equips Multimodal LLMs with an executable mechanism for "mental" visualization using SVG code.*
*   [Vision as Unified Multimodal Generation](papers/sensenova-vision.md) - *SenseNova-Vision expresses heterogeneous visual tasks in native text and image generation spaces, removing task-specific architectures.*
*   [LLM-as-a-Verifier: A General-Purpose Verification Framework](papers/llm-as-a-verifier.md) - *Introduces a general-purpose verification framework that computes continuous scores over scoring token logits to provide fine-grained feedback, improving verification scaling along score granularity, repeated evaluation, and criteria decomposition.*
*   [The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning](papers/mirage-optimizing-training-policies.md) - *Revealing training-inference mismatch in LLM RL post-training and introducing Monotonic Inference Policy Improvement (MIPI) to stabilize training.*
*   [Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming](papers/securing-the-ai-agent.md) - *An open-source red teaming framework (AI-Infra-Guard) for AI agents stratifying the attack surface across infrastructure, protocol, behavior, and model layers.*
* [ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning](papers/recontext-recursive-evidence-replay.md) - *A training-free inference method that uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation to improve long-context reasoning.*
*   [ASPIRE: Agentic /Skills Discovery for Robotics](papers/aspire-agentic-skills-discovery.md) - *A continual learning system that autonomously writes and refines robot control programs in a code-as-policy paradigm while compounding experience into a reusable skill library.*
* [BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding](papers/blockpilot-speculative-decoding.md) - *A sample-adaptive policy that predicts the optimal block size from the prefilling representation for diffusion-based speculative decoding, accelerating inference.*
*   [Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent](papers/agents-a1.md) - *A 35B Mixture-of-Experts Agentic Model that reaches trillion-parameter-level performance by scaling long-horizon trajectories.*
*   [BashCoder-R1: Towards Robust and Explainable Bash Code Generation with Robustness-Aware Group Relative Policy Optimization](papers/bashcoder-r1.md) - *A framework combining Continual Pre-training, Long Chain-of-Thought SFT, and Robustness-Aware GRPO to generate robust and explainable Bash scripts, outperforming DeepSeek-V3.2.*
*   [Qwen-Image-2.0-RL Technical Report](papers/qwen-image-2-0-rl.md) - *A post-training pipeline applying RLHF and on-policy distillation to improve visual quality and instruction-following for the Qwen-Image-2.0 diffusion model.*
*   [JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting](papers/jetspec-parallel-tree-drafting.md) - *Speculative decoding (SD) accelerates autoregressive Large Language Models (LLMs) by drafting multiple tokens and verifying them in parallel, but it faces a scaling limitation: increasing the draft budget improves speed only when acceptance remains high and drafting overhead stays low.*
*   [Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents](papers/progress-advantage-llm-agents.md) - *Process reward models enable fine-grained, step-level evaluation of LLMs, yet building them for agentic settings remains prohibitively difficult: long-horizon interactions, irreversible actions, and stochastic environment feedback make both human annotation and Monte Carlo estimation infeasible at scale.*
*   [Qwen-Image-Agent: Bridging the Context Gap in Real-World Image Generation](papers/qwen-image-agent.md) - *A unified agentic framework that treats user input as partial context and progressively constructs the generation context through Context-Aware Planning and Context Grounding.*
*   [Information-Aware KV Cache Compression for Long Reasoning](papers/infokv-cache-compression.md) - *InfoKV, an entropy-aware KV cache compression framework that incorporates information-theoretic signals rather than relying solely on attention weights.*
*   [The Verification Horizon: No Silver Bullet for Coding Agent Rewards](papers/verification-horizon-no-silver-bullet.md) - *A study revealing that generating complex candidate solutions is no longer difficult for modern coding agents, and that reliably verifying them has become the central challenge, requiring verification systems to co-evolve with generators.*
*   [Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do](papers/look-light-think-heavy.md) - *What can multimodal Chain-of-Thought reasoning do, and where and why does it fall short?*
*   [Are We Ready For An Agent-Native Memory System?](papers/agent-native-memory-system.md) - *A systematic experimental study of agent memory from a data management perspective.*
*   [NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers?](papers/naturebench.md) - *Evaluating AI coding agents beyond reproduction toward discovery on real scientific problems.*
*   [Qwen-AgentWorld: Language World Models for General Agents](papers/qwen-agentworld.md) - *Building foundation models for agentic environment simulation covering 7 domains via long chain-of-thought reasoning.*
*   [OpenThoughts-Agent: Data Recipes for Agentic Models](papers/openthoughts-agent.md) - *An open data curation pipeline yielding insights on the importance of task sources and diversity for training broadly capable agents.*
*   [When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents](papers/premature-commitment-in-llm-agents.md) - *Diagnosing long-horizon LLM agent failure where they settle on one reading of the evidence early, using cross-run hidden-state convergence.*
*   [A Verifiable Search Is Not a Learnable Chain-of-Thought](papers/verifiable-search-not-learnable-cot.md) - *Revealing that when a procedure's only solution is search over information-free structure, no faithful forward chain-of-thought exists to imitate.*
*   [Tapered Language Models](papers/tapered-language-models.md) - *An architectural principle in which a parameter-bearing component is monotonically tapered across depth under a fixed total budget.*
*   [Unlimited OCR Works](papers/unlimited-ocr-works.md) - *A model designed to emulate human parsing working memory using Reference Sliding Window Attention (R-SWA) to maintain a constant KV cache.*
*   [Self-Compacting Language Model Agents](papers/self-compacting-language-model-agents.md) - *A scaffold that allows the model itself to decide when and how to compact accumulated context using a compaction tool and a lightweight rubric.*
*   [PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models](papers/perceptiondlm.md) - *A multimodal diffusion language model that achieves parallel visual region perception and captioning using structured attention masking, bypassing autoregressive bottlenecks.*
*   [LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents](papers/ledgeragent.md) - *An inference-time method for tool-calling agents that maintains observed task states in a separate ledger.*
*   [Configurable Clinical Information Extraction with Agentic RAG: What Works, What Breaks, and Why](papers/acie-agentic-clinical.md) - *An on-premise agentic RAG pipeline that reasons over complete patient contexts.*
*   [Context-Aware RL for Agentic and Multimodal LLMs](papers/contextrl.md) - *A context-aware reinforcement learning method that improves long-horizon reasoning and multimodal performance through an indirect auxiliary objective.*
*   [FAPO: Fully Autonomous Prompt Optimization of Multi-Step LLM Pipelines](papers/fapo-fully-autonomous-prompt-optimization.md) - *A framework for fully autonomous prompt and structural optimization of multi-step LLM pipelines using an agent within a standardized codebase.*
*   [Thinking with Visual Grounding](papers/thinking-with-visual-grounding.md) - *A reasoning process where Vision-Language Models interleave natural-language thoughts with explicit point or box groundings of the visual evidence used at each step.*
*   [SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior](papers/sae-interventions-unreliable.md) - *Exposes a gap between feature-level control and behavioral completeness in Sparse Autoencoders.*
*   [Variable-Width Transformers](papers/variable-width-transformers.md) - *An architecture proposing nonuniform capacity allocation across network depth, outperforming uniform baselines.*
*   [LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling](papers/loopcoder-v2.md) - *Parallel Loop Transformers (PLT) that optimize test-time computation scaling by demonstrating the two-loop variant delivers broad gains over non-looped baselines.*
*   [Tangram: Unlocking Non-Uniform KV Cache Compression for Efficient Multi-turn LLM Serving](papers/tangram-non-uniform-kv-compression.md) - *A serving framework that handles non-uniform KV cache budgets across attention heads to improve multi-turn LLM serving throughput.*
*   [WebChallenger: A Reliable and Efficient Generalist Web Agent](papers/webchallenger.md) - *A generalist web agent framework achieving high performance without model scaling.*
*   [Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO](papers/s2l-po.md) - *S2L-PO framework leveraging small models as natural explorers to train larger models in GRPO.*
*   [The Cold-Start Safety Gap in LLM Agents](papers/cold-start-safety-gap.md) - *A study revealing LLM tool-calling agents are most vulnerable to safety threats at the start of sessions.*
*   [Evoflux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents](papers/evoflux-inference-time-evolution.md) - *An inference-time evolutionary search method that improves the execution reliability of small language models on complex tool workflows.*
*   [Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning](papers/switch-latent-reasoning.md) - *A framework named SWITCH that uses explicit boundary tokens to make latent chain-of-thought trainable via standard on-policy RL like GRPO.*
*   [Redesign Mixture-of-Experts Routers with Manifold Power Iteration](papers/manifold-power-iteration-moe.md) - *Aligning router rows with principal singular directions using Manifold Power Iteration.*
*   [LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers](papers/llmrouter.md) - *A unified infrastructure and evaluation pipeline (xRouteBench) for constructing, comparing, and deploying model routers to optimize inference cost and response quality.*
*   [InternVideo3: Agentify Foundation Models with Multimodal Contextual Reasoning](papers/internvideo3-agentify-foundation-models.md) - *A framework for long-video understanding via MCR and M^2LA.*
*   [Attention Amnesia in Hybrid LLMs: When CoT Fine-Tuning Breaks Long-Range Recall, and How to Fix It](papers/attention-amnesia-in-hybrid-llms.md) - *A foundational study and QK-Restore training-free fix for the long-context recall degradation caused by CoT fine-tuning.*
*   [SearchSwarm: Towards Delegation Intelligence in Agentic LLMs for Long-Horizon Deep Research](papers/searchswarm.md) - *A multi-agent paradigm that overcomes context exhaustion by training a main agent to intelligently decompose and delegate tasks to subagents.*
*   [AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing](papers/aha-wam.md) - *A decoupled, dual-frequency DiT architecture for high-performance robot manipulation.*
*   [Why Muon Outperforms Adam: A Curvature Perspective](papers/why-muon-outperforms-adam.md) - *An analysis showing Muon optimizes LLM training by maintaining lower Normalized Directional Sharpness.*
*   [Latent Reasoning with Normalizing Flows](papers/latent-reasoning-normalizing-flows.md) - *Introduces NF-CoT, a framework that models continuous thoughts with normalizing flows to replace discrete explicit Chain-of-Thought, preserving probabilistic sampling while reducing intermediate-reasoning cost.*
*   [Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings](papers/embedfilter-unembedding-matrix.md) - *Introduces EmbedFilter, a linear transformation that refines LLM text embeddings by filtering out the high-frequency token subspace encoded in the unembedding matrix, improving semantics and enabling dimensionality reduction.*
*   [Critic-R: Improving Agentic Search using Instruction-tuned Retrievers with Natural Language Introspective Feedback](papers/critic-r-agentic-search.md) - *A framework that improves agentic search by explicitly closing the feedback loop between the reasoning agent and the retrieval model using natural language introspective feedback, without requiring manual relevance annotations.*
*   [The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs](papers/shadow-price-of-reasoning.md) - *Introduces CLEAR, an optimal budget allocation framework that uses economic principles to dynamically distribute inference-time compute across queries, achieving up to a 3x improvement in global accuracy in resource-scarce regimes.*
* [Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution](papers/code2lora-hypernetwork-generated-adapters.md)
*   [AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization](papers/alphaq-calibration-free-moe-quantization.md) - *A novel, calibration-free mixed-precision quantization method for MoE models that leverages Heavy-Tailed Self-Regularization to achieve near full-precision accuracy with an average expert precision of only 3.5 bits.*
*   [EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Management](papers/evods-self-evolving-data-science-agent.md) - *A self-evolving autonomous data science agent that learns to expand its skills and adaptively managing long-term context through agentic reinforcement learning.*
*   [LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards](papers/longtracerl.md) - *A framework for long-context reasoning using tiered distractors and rubric rewards.*
*   [Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill](papers/skill-rm.md) - *A unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill, dynamically orchestrating heterogeneous resources.*
*   [Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalization](papers/positional-vs-symbolic-attention.md) - *An analysis of attention heads showing successful learning emerges as pure positional or symbolic heads, and their implications for length generalization.*
*   [Skill-Conditioned Gated Self-Distillation for LLM Reasoning](papers/skill-conditioned-gated-self-distillation.md) - *A robust gated self-distillation objective that treats teacher guidance as hypotheses to validate rather than unconditional rules to imitate.*
*   [PANDO: Efficient Multimodal AI Agents via Online Skill Distillation](papers/pando-online-skill-distillation.md) - *A single-rollout online skill-distillation framework that maintains a structured Skill Library and combines progress reflection, confidence-based skill demotion, hierarchical routing, visual compression, and cache-aware prompting.*
*   [CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM](papers/conf-kv-confidence-aware-eviction.md) - *A KV-cache manager that converts the next-token distribution into a scalar confidence score and uses it to choose the per-step cache budget, retaining more context when the model is uncertain and pruning aggressively when it is confident.*
*   [OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration](papers/omniverifier-m1.md) - *A generalist visual verifier leveraging symbolic meta-verification (like bounding boxes) and decoupled reinforcement learning for reliable multi-modal verification.*
*   [MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation](papers/muse-autoskill.md) - *A skill-centric agent framework for continuous improvement via lifecycle-managed skills.*
*   [LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding](papers/locateanything.md) - *A unified generative grounding and detection framework based on Parallel Box Decoding.*
-   [Looped Diffusion Language Models](papers/looped-diffusion-language-models.md) - *Selectively looping early-middle transformer layers in MDMs for training and inference efficiency.*
*   [SimSD: Simple Speculative Decoding in Diffusion Language Models](papers/simsd-speculative-decoding-diffusion.md) - *A plug-and-play masking strategy that equips diffusion language models with temporally valid token-level contexts for speculative decoding, achieving up to 7.46x higher decoding throughput.*
-   [OrpQuant: Geometric Orthogonal Residual Projection for Multiplier-Free Power-of-Two Transformer Quantization](papers/orpquant-transformer-quantization.md) - *Algorithm-hardware co-design for Power-of-Two (PoT) quantization addressing geometric limitations.*
*   [From Layers to Submodules: Rethinking Granularity in Replacement-Based LLM Compression](papers/subfit-submodule-replacement-compression.md) - *Introduces SubFit, a submodule-level compression method that selects non-contiguous Attention and FeedForward submodules, retaining 84.6% of dense downstream accuracy at 25% sparsity.*
-   [Language Models Need Sleep](papers/language-models-need-sleep.md) - *Sleep-like consolidation mechanism converting context into persistent fast weights using SSM blocks.*
*   [LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws](papers/shannon-scaling-law.md) - *A unified theoretical framework modeling LLM training as information transmission over a noisy channel, explaining catastrophic overtraining and quantization-induced degradation.*
*   [SkillOpt: Executive Strategy for Self-Evolving Agent Skills](papers/skillopt.md) - *A systematic controllable text-space optimizer for agent skills that turns scored rollouts into bounded add/delete/replace edits on a single skill document.*
*   [GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment](papers/golongrl.md) - *A fully open-source recipe for long-context RLVR featuring capability-oriented data construction and TMN-Reweight for heterogeneous multitask optimization.*
*   [MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems](papers/moss-self-evolution.md) - *A system that performs self-rewriting at the source level on production agentic substrates.*
*   [Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention](papers/gated-deltanet-2.md) - *Generalizing adaptive forgetting and channel-wise decay for Linear Attention without scrambling existing associations.*
*   [Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models](papers/incantation-natural-language-action.md) - *An interactive video world model that uses per-latent-frame natural language conditioning for multi-entity control, achieving 19.7 FPS at 480p.*
*   [MMSkills: Towards Multimodal Skills for General Visual Agents](papers/mmskills-towards-multimodal-skills-for-general-visual-agents.md) - *A framework for representing, generating, and using reusable multimodal procedures for runtime visual decision making.*
*   [Long Context Pre-Training with Lighthouse Attention](papers/lighthouse-attention.md) - *A symmetrical selection-based hierarchical attention algorithm that addresses the quadratic time and memory bottleneck of scaled dot-product attention in causal transformers during extreme long-context pre-training.*
*   [ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both](papers/atlas-agentic-latent-visual-reasoning.md) - *A framework combining agentic and latent visual reasoning using a single discrete functional token, stabilized by Latent-Anchored GRPO (LA-GRPO).*
*   [Multi-Turn On-Policy Distillation with Prefix Replay](papers/multi-turn-on-policy-distillation-prefix-replay.md) - *An off-environment on-policy distillation method for agentic tasks that uses pre-collected teacher trajectories as prefixes to avoid expensive online environment interactions.*
*   [SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation](papers/sana-video-2-hybrid-linear-attention.md) - *A hybrid video diffusion transformer combining linear attention and softmax anchors for efficient, high-resolution video generation on a single GPU.*
*   [NVIDIA-labs OO Agents: Native Python Object-Oriented Agents](papers/nvidia-nooa-object-oriented-agents.md) - *A model-agnostic Python framework (NOOA) that unifies agent development by representing agents as native Python objects, where methods act as actions and docstrings as prompts.*
*   [Self-Distilled Agentic Reinforcement Learning](papers/self-distilled-agentic-rl.md) - *A reinforcement learning framework that treats On-Policy Self-Distillation (OPSD) as a gated auxiliary objective, avoiding compounding multi-turn instability in multi-turn agents.*
*   [A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models](papers/single-neuron-safety-bypass.md) - *Safety alignment in LLMs can be bypassed by targeting a single refusal or concept neuron.*
*   [Teaching Language Models to Think in Code](papers/thinking-in-code.md) - *A framework replacing natural language reasoning with code execution to reduce intermediate reasoning errors.*
*   [FlashEvolve: Accelerating Agent Self-Evolution](papers/flashevolve-accelerating-agent.md) - *An efficient framework that replaces synchronized execution with asynchronous workers and queues to speed up LLM-based agent evolution.*
*   [LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling](papers/llms-improving-llms.md) - *AutoTTS: an environment-driven framework for automatic discovery of test-time scaling strategies.*
*   [UniPool: A Globally Shared Expert Pool for Mixture-of-Experts](papers/unipool-shared-expert-pool.md) - *An MoE architecture that replaces per-layer expert ownership with a single shared pool, reducing redundancy and enabling sublinear expert-parameter growth.*
*   [MiA-Signature: Approximating Global Activation for Long-Context Understanding](papers/mia-signature.md) - *A compact representation that approximates the global influence of activation on downstream processing for long-context tasks.*
*   [MARBLE: Multi-Aspect Reward Balance for Diffusion RL](papers/marble-reward-balance.md) - *A gradient-space optimization framework that maintains independent advantage estimators for each reward in Diffusion RL.*
*   [UniReasoner: Closing the Understanding-Generation Gap](papers/unireasoner.md) - *A framework leveraging LLMs as universal reasoners via visual drafts and self-critiques to guide diffusion models.*
*   [Flux Attention: Context-Aware Hybrid Attention](papers/flux-attention.md) - *A layer-level dynamic routing framework between Full and Sparse Attention for efficient LLM inference.*
*   [KellyBench: Evaluating Long-Horizon Sequential Decision Making](papers/kellybench.md) - *An environment simulating sports betting markets where frontier models struggle to achieve positive returns.*
*   [Exploration Hacking: Can LLMs Learn to Resist RL Training?](papers/exploration-hacking.md) - *A study revealing that capable LLMs can strategically alter their exploration during RL training to influence the outcome, highlighting a potential failure mode of alignment.*
*   [Step-Level Advantage Selection (SAS)](papers/step-level-advantage-selection.md) - *Stabilizing efficient reasoning by operating at the reasoning-step level to improve accuracy-efficiency trade-offs.*
*   [In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks](papers/in-context-prompting-obsoletes-agent-orchestration.md) - *A study revealing that putting the entire procedure in the system prompt and letting the model self-orchestrate significantly outperforms external orchestrator frameworks like LangGraph and CrewAI for procedural tasks.*
*   [How Do AI Agents Spend Your Money?](papers/token-consumption-agents.md) - *A systematic study revealing that agentic tasks are uniquely expensive and that frontier models struggle to predict token usage before task execution.*
*   [MathDuels: Evaluating LLMs as Problem Posers and Solvers](papers/mathduels.md) - *A self-play benchmark where models act as both authors and solvers of math problems.*
*   [Micro Language Models Enable Instant Responses](papers/micro-language-models.md) - *Ultra-compact models (8M-30M parameters) that instantly generate the first words of a response on-device, masking cloud latency.*
*   [EasyVideoR1: Easier RL for Video Understanding](papers/easyvideo-r1.md) - *A complete reinforcement learning framework tailored for training large vision-language models on video tasks.*
*   [CrossMath: Modality Gap in VLM Reasoning](papers/crossmath-modality-gap.md) - *A benchmark revealing that current Vision-Language Models primarily reason in text space and rely minimally on genuine visual evidence.*
*   [VGF: Reinforcement Learning via Value Gradient Flow](papers/vgf-reinforcement-learning.md) - *A scalable paradigm for behavior-regularized RL using optimal transport via discrete gradient flow.*
*   [DRA-GRPO: Diversity-aware Reward Adjustment for GRPO](papers/dra-grpo.md) - *A diversity-aware reward adjustment framework that prevents mode collapse in GRPO and enables high-quality mathematical reasoning.*
*   [RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomography](papers/radagent.md) - *A tool-using AI agent that generates CT reports through a stepwise and interpretable process.*
*   [RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time](papers/rational-rewards.md) - *A framework for teaching reward models to produce explicit critiques before scoring, improving generators during training and testing.*
*   [Relax: Omni-Modal RL Engine for MoEs](papers/relax-omni-modal-rl.md) - *An asynchronous, omni-native Reinforcement Learning post-training engine by RedNote AI for scaling post-training to multi-modal and MoE models.*
*   [Demystifying OPD: Length Inflation and Stabilization Strategies for LLMs](papers/demystifying-opd-length-inflation.md) - *Addressing truncation collapse in on-policy distillation with StableOPD.*
*   [PRISM: Pre-alignment via Black-box On-policy Distillation for Multimodal Reinforcement Learning](papers/prism-pre-alignment.md) - *Mitigating distributional drift in multimodal reinforcement learning by inserting an explicit distribution-alignment stage between SFT and RLVR.*
*   [Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts](papers/seeing-but-not-thinking-routing-distraction.md) - *Identifying and mitigating routing distraction that causes Multimodal MoE models to fail in reasoning tasks despite accurate visual perception.*
*   [TraceSafe: Systematic Assessment of LLM Guardrails on Multi-Step Trajectories](papers/tracesafe.md) - *A benchmark for assessing mid-trajectory safety in autonomous agents.*
*   [Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing (SRPO)](papers/sample-routed-policy-optimization.md) - *A unified on-policy framework combining GRPO and SDPO.*
*   [Friends and Grandmothers in Silico: Localizing Entity Cells](papers/friends-and-grandmothers-in-silico.md) - *Localizing entity-selective MLP neurons to study internal mechanisms of language models for factual queries.*
*   [Foundations of Schrödinger Bridges for Generative Modeling](papers/schrodinger-bridges-generative-modeling.md) - *A comprehensive toolkit for constructing Schrödinger bridges from first principles.*
*   [FlashAttention-4: Adapting to Asymmetric Hardware Scaling](papers/flash-attention-4.md) - *Algorithm and Kernel Pipelining Co-Design to handle shifting bottlenecks in Blackwell (B200) architectures.*
*   [AReaL: Asynchronous Reinforcement Learning](papers/areal.md) - *A fully asynchronous RL system decoupling generation from training to maximize GPU utilization.*
*   [Agent-as-a-Judge: Evaluate Agents with Agents](papers/agent-as-a-judge.md) - *Evaluating autonomous agents by using other agents to verify their process.*
*   [Vector Policy Optimization: Training for Diversity Improves Test-Time Search](papers/vector-policy-optimization.md) - *An RL algorithm (VPO) that explicitly trains policies to anticipate diverse downstream reward functions and produce diverse solutions for better test-time search.*

## Core Concepts & Architecture
*   [Odds Ratio Preference Optimization (ORPO)](concepts/orpo.md) - *A monolithic preference alignment algorithm that combines SFT and alignment without a reference model.*
*   [TurboQuant: Extreme KV Cache Compression](concepts/turboquant.md) - *Google's algorithm for near-optimal vector quantization, achieving absolute quality neutrality with 3.5 bits per channel.*
*   [DualPath: Agentic LLM Inference](concepts/dualpath.md) - *A system architecture by DeepSeek to overcome the storage bandwidth bottleneck.*
*   [1.58-bit LLMs: The Era of Ternary Weights](concepts/1-58-bit-llm.md) - *A new paradigm where LLM weights are restricted to {-1, 0, 1}, drastically reducing compute cost.*
*   [Flow Matching: Simulation-Free Continuous Normalizing Flows](concepts/flow-matching.md) - *A generative modeling paradigm that learns a continuous time-dependent transformation between noise and data without expensive simulations.*
*   [LatentMoE: Serving-Aware Sparse Compute](concepts/latent-moe.md) - *A memory-efficient MoE architecture.*
*   [DeepSeek Sparse Attention (DSA)](concepts/deepseek-sparse-attention.md) - *A sparse attention mechanism that reduces quadratic complexity in long-context LLMs while preserving quality.*
*   [Multi-Token Prediction (MTP)](concepts/multi-token-prediction.md) - *Predicting multiple future tokens in one forward pass.*
*   [StreamingLLM & Attention Sinks: Efficient Streaming Language Models](concepts/streaming-llm.md) - *Generalizing LLMs trained with finite context windows to infinite sequences without fine-tuning.*
*   [Continuous Batching: Iteration-Level Scheduling](concepts/continuous-batching.md) - *Maximizing GPU utilization by dynamically slotting in new requests as others finish.*
*   [State Space Models (SSM) & Mamba: The Linear-Time Architecture](concepts/state-space-models.md) - *An alternative to Transformers that solves the quadratic attention bottleneck via selective state spaces.*
*   [Causal vs. Masked Language Models](concepts/causal-vs-masked-language-models.md) - *The architectural divide between generative decoder-only models (like GPT) and bidirectional encoder-only models (like BERT).*
*   [PagedAttention: Eliminating KV Cache Waste](concepts/paged-attention.md) - *A memory management technique that borrows from virtual memory concepts to dramatically improve LLM inference throughput.*
*   [Reinforcement Learning from Human Feedback (RLHF)](concepts/rlhf.md) - *A technique to align an intelligent agent with human preferences by training a reward model to score responses and optimize the policy.*
*   [Direct Preference Optimization (DPO)](concepts/direct-preference-optimization.md) - *A lightweight alternative to RLHF that turns the language model itself into a reward model via contrastive pairs.*
*   [Group Relative Policy Optimization (GRPO): Efficient Reinforcement Learning](concepts/group-relative-policy-optimization.md) - *Streamlining RL by eliminating the value model and using group-based reward baselines.*
*   [Low-Rank Adaptation (LoRA): Efficient Fine-Tuning](concepts/low-rank-adaptation.md) - *Fine-tuning massive models efficiently by injecting small, trainable rank decomposition matrices.*
*   [Weight-Decomposed Low-Rank Adaptation (DoRA)](concepts/dora-weight-decomposed-low-rank-adaptation.md) - *A variant of LoRA that decomposes the pre-trained weight into two components, magnitude and direction, to enhance learning capacity without adding inference overhead.*
*   [Grouped-Query Attention (GQA)](concepts/grouped-query-attention.md) - *An optimization technique that balances the efficiency of Multi-Query Attention with the performance of Multi-Head Attention.*
*   [Retrieval-Augmented Generation (RAG): Bridging Knowledge and Generation](concepts/retrieval-augmented-generation.md) - *Reducing hallucinations by grounding model outputs in external data.*
*   [Transformer Architecture: The Engine of Modern AI](concepts/transformer-architecture.md) - *The core mechanism behind GPT, BERT, and most modern LLMs using Self-Attention.*
*   [Chain-of-Thought (CoT): Eliciting Reasoning in LLMs](concepts/chain-of-thought.md) - *How models solve complex problems by "thinking" before speaking.*
*   [Mixture of Experts (MoE): Scaling Parameters Without Scaling Costs](concepts/mixture-of-experts.md) - *How models like Mixtral and DeepSeek scale efficiently.*
*   [Inference-Time Compute: The New Scaling Law](concepts/inference-time-compute.md) - *Shift from training-time scale to test-time reasoning (System 2 thinking).*
*   [Rotary Position Embedding (RoPE)](concepts/rotary-position-embedding.md) - *Injecting relative positional information by mathematically rotating token embeddings in space.*
*   [Quantization: Shrinking Models for the Real World](concepts/quantization.md) - *Reducing model weights to lower precision formats (e.g., INT4) to drastically cut VRAM requirements with minimal accuracy loss.*
*   [FlashAttention: IO-Aware Exact Attention](concepts/flash-attention.md) - *A memory-efficient algorithm that reduces HBM/SRAM access overhead, turning quadratic memory complexity into linear.*
*   [Multi-Head Latent Attention (MLA): Compressing the KV Cache](concepts/multi-head-latent-attention.md) - *A low-rank factorization technique that shrinks KV cache size drastically, enabling longer context sizes and faster inference in DeepSeek models.*
*   [Ring Attention: Near-Infinite Context through Distributed Compute](concepts/ring-attention.md) - *A blockwise attention technique that scales context sizes linearly with the number of GPUs by forming a logical ring to overlap computation and communication.*

## Prompt Engineering Techniques
*   [Visual prompt engineering for video models](techniques/visual-prompt-engineering.md) - *A technique for automatically modifying task images to improve visual reasoning performance in video models, often more effective than text-based prompt engineering.*
*   [Experience Augmented Policy Optimization for LLM Reasoning](techniques/eapo.md) - *Experience-Augmented Policy Optimization (EAPO) leverages a prior RL-optimized policy as an action-level experience prior and selectively injects experience at critical decision points during rollout.*
*   [Token-Level Policy Optimization (TLPO)](techniques/token-level-policy-optimization.md) - *A fine-tuning framework designed to mitigate language confusion in LLMs through localized, token-level updates.*
*   [Context Caching: The Cost-Cutting Superpower](techniques/context-caching.md) - *Reducing cost and latency by reusing the KV cache for large prompts.*
*   [Double Question Prompting: Overcoming Causal LLM Limitations](techniques/double-question-prompting.md)
*   [Speculative Decoding](techniques/speculative-decoding.md) - *Accelerating LLM inference by pairing a small draft model with a larger verification model.*

## Telegram
* [CleveresTech Telegram Group](t.me/cleverestech)

## Tools & Protocols
*   [OpenAI Symphony: Open-Source Agentic Framework](news/openai-symphony.md) - *An open-source, Elixir-based framework designed to manage autonomous AI coding agents through structured implementation runs.*
*   [Sema Code: Decoupling AI Coding Agents into Programmable, Embeddable Infrastructure](news/sema-code.md) - *An open AI coding framework that decouples the core agent engine into a standalone library.*
*   [QonQrete: Sandboxed Agentic IDE](news/qonqrete.md) - *A local-first multi-agent system for code generation in safe sandboxes.*
*   [Claude Code: Anthropic's Agentic CLI Assistant](news/claude-code.md) - *Anthropic's CLI tool for autonomous coding and terminal commands, powered by Claude 3.7 Sonnet.*
*   [Dynatomics: Larry Page's AI Manufacturing Startup](news/dynatomics.md) - *A new AI-first manufacturing company aiming to optimize product design and production using LLMs.*
*   [Model Context Protocol (MCP): The USB-C for AI](concepts/model-context-protocol.md) - *Standardizing how AI models connect to data and tools, with Context7 as a prime example.*

cleveres-ai

cleveres-ai 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

cleveres-ai

cleveres-ai 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README