X / Twitter
Kokoro is insane. 🤯
This AI is a groundbreaking TTS model with just 82M parameters. It outperforms larger models and generates minutes of speech in seconds.
APACHE 2.0 licensed, trained on < 100 hours of audio is now available in ai-gradio
pip install --upgrade "ai-gradio[kokoro]"
import gradio as gr
import ai_gradio
gr.load(name='kokoro:kokoro-v0_19', src=ai_gradio.registry,).launch()
🚨Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks
🌟𝐏𝐫𝐨𝐣: https://t.co/7rByUl47OB
A unified multimodal large language model that integrates region-level understanding for both images and videos. Given user-defined localized region inputs (boxes or masks) accompanied by a corresponding text prompt, Omni-RGPT generates responses tailored to the visual context of each region for both images and videos.
Incredible book for teaching creative writing. Intended for a classroom, but easily adaptable to homeschool. The kids and I are having lots of fun. https://t.co/YbURx96aUl
DiMA: distilling a VLM into a planner for driving.
They use VAD as a baseline and pass tokens from its models into an LLM. The LLM is trained to do VQA, MAE, future prediction, and scene editing. You distill the LLM into planner transformer--you don't need the LLM for inference. https://t.co/b1nPAxsI0M
Preprint of the day: Motamed et. al, "Physics IQ Benchmark: Do generative video models learn physical principles from watching videos?" -- https://t.co/QwEXrXMyo1
A benchmark to test whether video models really learned physics -- spoiler: they didn't (yet) https://t.co/bfG8W8NQoj
The later features in DINO-v2 are more abstract and semantically meaningful than I'd expected from the training objectives.
This neuron responds only to hugs. Nothing else, just hugs. https://t.co/FVzMVWpQB0
Foundations of LLMs
This amazing new LLM book just dropped on arXiv.
200+ pages!
It covers areas such as pre-training, prompting, and alignment methods.
It looks like a great intro to LLMs for devs and researchers.
A review in 23 tweets.
The first three chapters of @astupple’s book argue, amongst other things, that children should be allowed to eat ice cream and oreos, as their staple foods, as often as and for as long as they desire, and have unlimited iPad time with no YouTube limits. https://t.co/HgxYZn11JI
🪶 MagicQuill
An Intelligent Interactive Image Editing System https://t.co/vfUMp0CsK3
SynthLight
Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces https://t.co/PnX0j4r8mN
Dear Readers, you'll ❤️ this: Introducing ReaderLM-v2, a 1.5B small language model for HTML-to-Markdown conversion and HTML-to-JSON extraction with exceptional quality. Thanks to the new training paradigm and higher-quality training data, ReaderLM-v2 is a significant leap forward from its predecessor, particularly in handling long-context and markdown syntax. While the first generation approached HTML-to-markdown conversion as a "selective-copy" task, v2 treats it as a true translation process. This enables the model to masterfully leverage markdown syntax, excelling at generating complex elements like code fences, nested lists, tables and LaTex equations. You can use ReaderLM-v2 today via Reader API, HuggingFace, AWS Sagemaker, etc.
given a massive list of tool calls, you probably want to break them up into domains and have specific agents handling them, so your tool call list becomes just a list of other agents to pass on the task to, increases accuracy for the cost of an extra llm call
(this is probably well known, but I've been working on implementing it all from first principles and rediscovering common knowledge, enjoying it)