multimodal ai

Bilingual

thinkingmachines/Inkling-Small
2 months ago
- Inkling-Small is a general-purpose multimodal model that accepts text, image, and audio inputs and generates text outputs.
- It can be used with libraries like Transformers, vLLM, SGLang, and Docker, as well as through APIs and third-party providers.
- The model uses a 42-layer decoder-only transformer with a sparse Mixture-of-Experts (MoE) architecture, totaling 276B parameters but activating only 12B per token.
- Training data is sourced from public, third-party, and synthetic data, with cleaning and filtering to improve quality and safety.
- Evaluations show strong performance across agentic, reasoning, factuality, vision, audio, and safety benchmarks.
- Safety mitigations include red-teaming and refusal suppression testing; residual risks like compliance with harmful role-play are addressed with defense-in-depth and moderation tools.
- Limitations include hallucination, biases, uneven multilingual performance, and knowledge cutoff; human oversight is recommended for high-stakes applications.
- The model is released with open weights for research, fine-tuning, and integration into third-party products.
Flux 3 X Mimic: The Next Generation of Video-Action Models
2 months ago
- FLUX 3 is a multimodal foundation model that generates audio-visual content and serves as the backbone for FLUX-mimic video-action models used in robotics.
- Video prediction is the most computationally intensive part, accounting for over 95% of training costs, as it forces the model to learn physical world dynamics like contact and motion.
- Adding action prediction to FLUX 3 initially degraded video quality by 10%, but after 3500 training steps, the model recovered full performance while also predicting actions.
- FLUX-mimic, built on FLUX 3, decodes actions from intermediate features of the video prediction path using a lightweight action decoder, achieving state-of-the-art success rates in robotic manipulation.
- The Self-Flow technique unifies generation and representation learning, improving both world model quality and downstream task performance for robotics.
- FLUX-mimic offers up to 10x sample efficiency over vision-language-action models and can run on a single NVIDIA RTX 5090 GPU with latency under 80ms for world representation.
- The model has been deployed in real factory settings at Audi, handling tasks like kitting, assembly, and handling soft materials that conventional automation couldn't manage.
- FLUX-mimic naturally recovers from task failures, such as missed grasps, without needing explicit demonstrations for every error scenario.
Inkling: Our Open-Weights Model
3 months ago
- The company has released a new AI model called Inkling, which is a Mixture-of-Experts transformer with 975B total parameters and 41B active parameters.
- Inkling supports a context window of up to 1 million tokens, was pretrained on 45 trillion tokens of multimodal data, and is available for fine-tuning via the Tinker platform.
- Key features of Inkling include multimodal reasoning (text, images, audio), controllable thinking effort for balancing performance and efficiency, and strong performance across reasoning, agentic coding, and safety benchmarks.
- Alongside Inkling, a preview of Inkling-Small (276B total parameters, 12B active) is shared, offering similar capabilities with lower cost and latency for certain workloads.
- The model's development involved large-scale RL training (over 30M rollouts), which improved reasoning and led to more concise reasoning styles.
- Inkling is designed as a broad foundation model for customization, emphasizing calibration, instruction following, and resistance to censorship (epistemics).
- Safety evaluations show Inkling has strong safeguards, performing well on benchmarks like FORTRESS and StrongREJECT.
- Inkling is available through various platforms and partners, including TogetherAI, Hugging Face, and others, with full weights released for customization.
Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone
3 months ago
- Bonsai 27B is a new multimodal model from the Bonsai family, based on Qwen3.6 27B, and the first of its capability class to run on a phone.
- It comes in two variants: Ternary Bonsai 27B (5.9 GB) for quality on laptops and 1-bit Bonsai 27B (3.9 GB) for footprint on phones, both using low-bit weights end-to-end with no high-precision escapes.
- The model retains 95% (Ternary) and 90% (1-bit) of full-precision baseline performance across benchmarks, excelling in math, coding, and tool calling for agentic workloads.
- Bonsai 27B enables local execution for agentic AI, reducing costs and enhancing privacy by keeping data on-device, and supports hybrid deployments with cloud models.
- It achieves high speeds (up to 163 tok/s on RTX 5090) and fits phone memory constraints, with a 262K-token context, multimodal vision, and speculative decoding.
- The release represents a paradigm shift in intelligence density, advancing AI deployment on everyday devices under the Apache 2.0 License, with plans for larger models.
Introducing Muse Spark 1.1
3 months ago
- Meta launches Muse Spark 1.1, an upgraded multimodal reasoning model for agentic tasks with improved tool/computer use, coding, and multimodal understanding.
- The model features a 1-million token context window, multi-agent orchestration for faster project handling, and public preview via the Meta Model API.
- Muse Spark 1.1 excels in computer-use workflows, coding (debugging, migrations), multimodal perception, and safety, with strong adversarial robustness.
- Early partners like Replit and Box praise its comprehensive capabilities for large-scale agentic workloads and enterprise applications.
- Meta highlights ongoing research momentum, with more advanced models in training, advancing toward personal superintelligence.
ThinkingCap-Qwen3.6-27B: Qwen3.6 capabilities with 50% fewer thinking tokens
3 months ago
- The ThinkingCap-Qwen3.6-27B model is a finetuned version of Qwen3.6-27B, designed to reduce thinking tokens by 50% on average while maintaining answer quality.
- It can be used with libraries like Transformers and vLLM, and platforms such as Google Colab, Kaggle, and Docker for local deployment.
- Out-of-domain evaluations show token reductions of up to 67.8% on benchmarks like GPQA-Diamond, with maintained accuracy in areas like knowledge, math, and multimodal tasks.
- In-domain evaluations on holdout datasets like GSM8K and ARC-Challenge demonstrate even higher token efficiency, with reductions up to 74.1% and improved accuracy.
- Safety benchmarks indicate preserved guardrails, with refusal rates similar to the base model and token reductions of around 20-24% on safety prompts.
- The model supports GGUF quantization for efficient local inference via llama.cpp and compatible runtimes, with quantized versions available in a sibling repository.
- Usage examples include image-text-to-text tasks, and the model requires specific sampling parameters (e.g., temperature=1.0) as recommended for optimal performance.
Gemma 4 on Cerebras - The Fastest Inference Is Now Multimodal
3 months ago
- Gemma 4 31B runs at over 1,800 tokens per second on Cerebras Inference, enabling fast multimodal applications.
- It is the first Google DeepMind model on Cerebras with image input support for screenshots, documents, charts, and UI states.
- Cerebras achieves record speed and low latency, making Gemma 4 suitable for real-time visual and agentic workflows.
- Gemma 4 31B matches Claude Haiku 4.5 in intelligence but runs 18x faster and is open-weight under Apache 2.0.
- The model's high speed transforms product development, allowing instant iteration and complex multimodal loops.
- As a dense, efficient model, Gemma 4 31B is designed for quality without the large footprint of MoE models.
- Example applications include real-time screenshot analysis, long-context summarization, and UI debugging.
- Gemma 4 31B is available on the Cerebras Inference Cloud in public preview for multimodal workloads.

multimodal ai

thinkingmachines/Inkling-Small

Flux 3 X Mimic: The Next Generation of Video-Action Models

Inkling: Our Open-Weights Model

Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone

Introducing Muse Spark 1.1

ThinkingCap-Qwen3.6-27B: Qwen3.6 capabilities with 50% fewer thinking tokens

Gemma 4 on Cerebras - The Fastest Inference Is Now Multimodal