October had many language model releases. Mid-size models, and even small models, are catching up in performance to borderline models like the GPT-4.5o. But the release that blew us all away wasn’t the language model: it was Claude’s desktop API. Using a computer allows you to teach Claude how to use a computer: how to run an application, click buttons, and use a shell or editor. It has many problems, security not the least of which – but it needs to improve. Sending screenshots to Claude so it can calculate where to click is clumsy at best, and there are undoubtedly better solutions (like using accessibility tools). However, using a computer gives us a glimpse into a future where we will work with agents who can plan and execute complex multi-step operations.
AI
- Little Language Models is an educational program that teaches young children about probability, artificial intelligence and related topics. It’s fun and playful and allows kids to build simple models of their own.
- Grafana and NVIDIA are working on a large language model for observability, apparently awkwardly named LLo11yPop. The goal of the model is to answer natural language questions about system health and performance based on telemetry data.
- Google is open-sourcing SynthID, a text watermarking system, so AI-generated documents can be traced back to the LLM that generated them. Watermarks do not affect the accuracy or quality of the generated documents. SynthID watermarks resist some manipulation, including editing.
- Mistral released two new models, the Ministral 3B and the Ministral 8B. These are small models designed to work on “edge” systems with limited resources. Unlike many previous small Mistral models, these are not open source.
- Anthropic added an API to Claude to “use the computer”. Using a computer allows the model to take control of the computer and use it to search for data by reading the screen, clicking buttons and other means, and typing. It is currently in beta.
- Moonshine is a new open source speech-to-text model that has been optimized for small devices with limited resources. Claims Whisper-equivalent accuracy at five times the speed.
- Meta is releasing a free dataset called Open Materials 2024 to help materials scientists discover new materials.
- Anthropic has published some tools for working with Claude on GitHub. At this point, tools are available to help analyze financial data and create customer support agents.
- NVIDIA has quietly launched Llama-3.1-Nemotron-70B-Instruct-HF, a language model that outperforms both GPT-4o and Claude 3.5 in benchmarks. This model is based on the open source Llama and is relatively small (70B parameters).
- NotebookLM got everyone excited with its ability to generate podcasts. Google has taken this a step further by adding tools that give users more control over what virtual podcast participants say.
- Data literacy is the new survival skill: We’ve known this for a while, but it’s all too easy to forget, especially in the age of artificial intelligence.
- The Open Source Initiative has a “modest” definition for open source AI. The definition recognizes four different categories of data: open, public, reachable, and unshareable.
- Do training AI models require huge data centers? PrimeIntellect trains the 10B model using distributed resources.
- OpenAI has published Swarm, a platform for building AI agents, on GitHub. They note that Swarm is experimental and will not respond to download requests. Feel free to join the experiment.
- OpenAI also released Canvas, an interactive tool for writing code and text using GPT-4o. Canvas is similar to Claude’s Artifacts.
- Two of the newly released Llama 3.2 models – the 90B and 11B – are multimodal. Model 11B will run comfortably on a laptop. Meta also released the Llama Stack API, a set of APIs that help developers build generative AI applications.
- OpenAI has announced a pseudo-real-time API. They aim to enable the creation of realistic voice applications, including the ability to interrupt AI in the flow of a conversation.
- Will artificial intelligence glasses become the next consumer blockbuster? Meta’s Orion prototype could be a killer user interface for AI. It’s not about playing; it’s about asking the AI about things you see. Now if they can only be produced at a decent price.
- AI avatars interview job applicants. This is not going to go well…
- The Allen Institute has developed a small language model called Molmo, which it claims has performance equivalent to GPT-4o.
- Humane Intelligence, an organization founded by Rumman Chowdhury, has offered a prize to developers building an AI vision model that can detect hate-based images online.
- It’s no surprise these days that chess and other board games can be played on a computer. Purpose table tennis? You may prefer video over paper.
- The Qwen family of language models with parameters from 0.5B to 72B are getting impressive reviews. Even the biggest ones can be made to work on older GPUs, not just the H100 and A100.
- Now AI can “prove” it’s human. An AI-based computer vision model has demonstrated the ability to beat Google’s latest CAPTCHA (reCAPTCHAv2) 100% of the time.
- Open AI now extends access to advanced voice mode for multiple users. Thanks to the advanced voice mode, ChatGPT is truly conversational: You can interrupt it in the middle of a sentence and it responds to your tone of voice.
- Neural motion planning is a neural network-based technique that allows robots to plan and execute tasks in unfamiliar environments.
Programming
- Safe C++ proposes an extension to C++ to be memory safe. Flaws in memory security have long been the biggest source of security bugs.
- Microsoft sees GenAIOps as a “paradigm shift” for IT. It will continue to be needed as software embraces AI and IT teams must become AI infrastructure specialists. One aspect of GenAIOps will be the collection, management and cleansing of datasets.
- Huly is an open source project management platform.
- Typst is a new system for writing scientific (and other) texts. It has capabilities equivalent to LaTeX, but the syntax is much simpler, similar to Markdown.
- Microsoft has started a project to make Linux eBPF available on Windows. In the Linux world, eBPF has proven to be an invaluable monitoring, security and compliance tool. Windows eBPF will be bytecode compatible with Linux.
- Python 3.13 is out. The most important changes are a new REPL that includes multiline editing and color support; experimental option to disable global interpreter lock (GIL); and an experimental just-in-time compiler.
- Ziggy is a new language for data serialization. It is not a general-purpose programming language; it is a specialized language for precisely and painlessly defining data schemas.
- Microsoft’s new security initiative is tied to their platform development efforts. Platform engineering limits the number of tools developers must use, which in turn reduces the amount of code that needs to be secured and maintained.
- The CNCF Artifact Hub is a resource for cloud-native configurations, plug-ins, and other software for building cloud-native infrastructure. It’s not a GitHub-like repository; refers back to artifact resources rather than storing them.
- Want to run Linux on an Intel 4004, a CPU from 1971? The launch will take almost 5 days. What’s more amazing is that it actually runs on an emulator that runs on a 4004.
Security
- It’s no surprise that fast injection works well against Anthropic’s amazing computing API. Anthropic’s documentation warns of many vulnerabilities. So it’s no wonder someone went ahead and tried it. Keep experimenting, but be careful.
- Imprompter is an attack against large language models that uses a malicious prompt to force the model to exfiltrate data from previous chats.
- One of the main sources of security errors is code that contains secrets (account names and passwords, certificates, etc.). HashiCorp’s Vault Radar scans software, including repositories and download requests, to detect secrets that have been exposed.
- Security researchers at Mandiant found that 70% of the vulnerabilities exploited last year were zero-days – new vulnerabilities that had not been previously reported. Once vulnerabilities are discovered, they are almost immediately weaponized and used for attacks.
- OpenAI has shut down the accounts of threat actors who use GPT for a range of activities, including malware development, disinformation generation and dissemination, and phishing. It would be surprising if similar abuse did not occur in other models.
- GitLab’s latest security update addresses a vulnerability that could allow attackers to run CI/CD pipelines on any repository branch.
- Students linked Meta’s Ray-Ban smart glasses to an invasive image search site. They then use language models to assemble data from a number of databases that contain personal information such as addresses.
- Cloudflare blocked a number of distributed denial-of-service (DDoS) attacks, including one that peaked at 3.8 terabits per second, the highest on record.
- In incident reviews, don’t talk about actions that respond to the incident. Incident control is about learning and understanding; talking about repairs will derail it. Corrections can always be discussed later and it is better if they are based on a firm understanding.
- We’ve known for a long time that requests to change passwords were bad practice. NIST is now proposing rules that would remove password composition requirements such as one uppercase letter, one number, and one non-Latin character.
- An immediate GPT long-term storage attack allows an attacker to send all user input and output to any server. This attack is permanent; it remains in GPT long-term memory. There has been a partial fix at this point.
- Kaspersky, which is discontinuing operations in the US, deleted their software from US users’ computers and installed Pango Group’s UltraAV and (in some cases) UltraVPN without users’ permission. Kaspersky’s behavior begs the question: When does an antimalware vendor become malware?
Web
- Videos from XOXO 2024 have been released. Molly White and Erin Kissane are especially highly recommended.
- Do we still need another React web framework? The One developers think so. One promises to be simple, headstrong and, above all, local.
- Tom Coates announced the creation of the Social Web Foundation, an organization dedicated to helping federated networks grow in a healthy way.
- Trouble in the WordPress world: WordPress.org has blocked WP Engine, an important hosting provider for WordPress users, from accessing its resources. Drama ensues, escalating and getting worse.
Hardware
Biology
Learn faster. Dig deeper. See further.
