Zebeth Media Solutions

Stability AI

Movio wants to make your marketing videos with generative AI • ZebethMedia

Generative AI is suddenly everywhere. Over the past year, you’ve probably seen people showcasing impressive AI-generated artworks, thanks to progress in text-to-image algorithms introduced by groups like OpenAI and Stability AI. A proliferation of startups is now trying to devise applications for this new class of language model, where the machine is capable of creating new text, images, and videos based on simple human input. One of them is Movio, a two-year-old startup leveraging generative AI along with other machine learning frameworks like GAN to make videos featuring talking human avatars. The platform is going after marketers with a Canva-style drag-and-drop interface. Users will first pick from a range of templates, be it a theme for a shopping site or a trip to Japan. Then they can add a hyperrealistic avatar to be the video’s “spokesperson”, with speech generated by text input. The outfit, face, and voice of the AI-made human can be swapped with a click. Movio’s user base is currently in the lower hundreds of thousands, with paying customers nearing 1,000. It has so far raised around $9 million in funding from investors including IDG, Sequoia Capital China, and most recently, Baidu Ventures. Xu met his co-founder and CFO Liang Wang, a veteran of ByteDance and the music social network Smule, when the two were studying at Carnegie Mellon University. Last year, we covered how Movio, which was then called Surreal, landed a brilliant use case for deepfake. At the time, the company was based in Shenzhen, the hardware haven also known for its vibrant export-led e-commerce industry — most of Amazon’s sellers are from the metropolis. Merchants were using Movio to create promo videos narrated by synthesized humans, doing away with the need to hire real models. Movio recently moved to Los Angeles, where its co-founder and CEO Josh Xu previously worked for six years as a Snap engineer. That’s because the startup is hoping to capture the wave of marketers who are warming up to AI tools to assist their job. “We are doing what Jasper and Copy.ai do but for video production,” Xu said to ZebethMedia, referring to two of the top AI content helpers of today. “Videos are powerful — just image if marketers can send emails with talking human avatars instead of plain text.” Movio can only synthesize talking heads for now, but it’s working toward a future where its algorithms can generate whole-body movement, which will allow the company to get closer to its goal of being an “all-in-one AI video production platform.” The startup charges users by the length of videos, which is correlated with the script they submit, as well as a premium fee from those who use customized faces, a feature that is particularly popular for “corporate training,” according to Xu. Movio has also opened its API to third-party websites, some of which are using its engine to create pop-up customer support avatars. “AI-generated video is just a small segment within the AIGC [AI generated content] industry. We’ve seen how much text-to-image can do, and I expect text-to-video to bring about even greater disruption when it’s ready,” said Xu.

Stability AI backs effort to bring machine learning to biomed • ZebethMedia

Stability AI, the venture-backed startup behind the text-to-image AI system Stable Diffusion, is funding a wide-ranging effort to apply AI to the frontiers of biotech. Called OpenBioML, the endeavor’s first projects will focus on machine learning based approaches to DNA sequencing, protein folding, and computational biochemistry. The company’s founders describe OpenBioML as an “open research laboratory” — aims to explore the intersection of AI and biology in a setting where students, professionals and researchers can participate and collaborate, according to Stability AI CEO Emad Mostaque. “OpenBioML is one of the independent research communities that Stability supports,” Mostaque told ZebethMedia in an email interview. “Stability looks to develop and democratize AI, and through OpenBioML, we see an opportunity to advance the state of the art in sciences, health and medicine.” Given the controversy surrounding Stable Diffusion — Stability AI’s AI system that generates art from text descriptions, similar to OpenAI’s DALL-E 2 — one might be understandably wary of Stability AI’s first venture into health care. The startup has taken a laissez-faire approach to governance, allowing developers to use the system however they wish, including for celebrity deepfakes and pornography. Stability AI’s ethically questionable decisions to date aside, machine learning in medicine is a minefield. While the tech has been successfully applied to diagnose conditions like skin and eye diseases, among others, research has shown that algorithms can develop biases leading to worse care for some patients. An April 2021 study, for example, found that statistical models used to predict suicide risk in mental health patients performed well for white and Asian patients but poorly for Black patients. OpenBioML is starting with safer territory, wisely. Its first projects are: BioLM, which seeks to apply natural language processing (NLP) techniques to the fields of computational biology and chemistry DNA-Diffusion, which aims to develop AI that can generate DNA sequences from text prompts LibreFold, which looks to increase access to AI protein structure prediction systems similar to DeepMind’s AlphaFold 2 Each project is led by independent researchers, but Stability AI is providing support in the form of access to its AWS-hosted cluster of over 5,000 Nvidia A100 GPUs to train the AI systems. According to Niccolò Zanichelli, a computer science undergraduate at the University of Parma and one of the lead researchers at OpenBioML, this will be enough processing power and storage to eventually train up to ten different AlphaFold 2-like systems in parallel. “A lot of computational biology research already leads to open-source releases. However, much of it happens at the level of a single lab and is therefore usually constrained by insufficient computational resources,” Zanichelli told ZebethMedia via email. “We want to change this by encouraging large-scale collaborations and, thanks to the support of Stability AI, back those collaborations with resources that only the largest industrial laboratories have access to.” Generating DNA sequences Of OpenBioML’s ongoing projects, DNA-Diffusion — led by pathology professor Luca Pinello’s lab at the Massachusetts General Hospital & Harvard Medical School — is perhaps the most ambitious. The goal is to use generative AI systems to learn and apply the rules of “regulatory” sequences of DNA, or segments of nucleic acid molecules that influence the expression of specific genes within an organism. Many diseases and disorders are the result of misregulated genes, but science has yet to discover a reliable process for identifying — much less changing — these regulatory sequences. DNA-Diffusion proposes using a type of AI system known as a diffusion model to generate cell-type-specific regulatory DNA sequences. Diffusion models — which underpin image generators like Stable Diffusion and OpenAI’s DALL-E 2 — create new data (e.g. DNA sequences) by learning how to destroy and recover many existing samples of data. As they’re fed the samples, the models get better at recovering all the data they had previously destroyed to generate new works. Image Credits: Stability AI “Diffusion has seen widespread success in multimodal generative models, and it is now starting to be applied to computational biology, for example for the generation of novel protein structures,” Zanichelli said. “With DNA-Diffusion, we’re now exploring its application to genomic sequences.” If all goes according to plan, the DNA-Diffusion project will produce a diffusion model that can generate regulatory DNA sequences from text instructions like “A sequence that will activate a gene to its maximum expression level in cell type X” and “A sequence that activates a gene in liver and heart, but not in brain.” Such a model could also help interpret the components of regulatory sequences, Zanichelli says — improving the scientific community’s understanding of the role of regulatory sequences in different diseases. It’s worth noting that this is largely theoretical. While preliminary research on applying diffusion to protein folding seems promising, it’s very early days, Zanichelli admits — hence the push to involve the wider AI community. Predicting protein structures OpenBioML’s LibreFold, while smaller in scope, is more likely to bear immediate fruit. The project seeks to arrive at a better understanding of machine learning systems that predict protein structures in addition to ways to improve them. As my colleague Devin Coldewey covered in his piece about DeepMind’s work on AlphaFold 2, AI systems that accurately predict protein shape are relatively new on the scene but transformative in terms of their potential. Proteins comprise sequences of amino acids that fold into shapes to accomplish different tasks within living organisms. The process of determining what shape an acids sequence will create was once an arduous, error-prone undertaking. AI systems like AlphaFold 2 changed that; thanks to them, over 98% of protein structures in the human body are known to science today, as well as hundreds of thousands of other structures in organisms like E. coli and yeast. Few groups have the engineering expertise and resources necessary to develop this kind of AI, though. DeepMind spent days training AlphaFold 2 on tensor processing units (TPUs), Google’s costly AI accelerator hardware. And acid sequence training data sets are often proprietary or released under non-commercial licenses. Proteins folding

Subscribe to Zebeth Media Solutions

You may contact us by filling in this form any time you need professional support or have any questions. You can also fill in the form to leave your comments or feedback.

We respect your privacy.
business and solar energy