Lip Syncing with A.I is now easy!

Both the Alibaba group and Pika came out with some revolutionary technology to allow your pictures to animate and say what you want them to in a simple way.

Welcome back,

Another insane week in A.I. Announcements and news.

Let’s dive in!

The Rundown

  • FEATURED

    • You can now use lip sync in all of your A.I. Videos

  • NEWS

    • Adobe introduces a new GenAI Music Tool

    • Google has more bad press

    • Microsoft partners with Mistral

    • Meta making strategic partnerships

    • Apple is shifting gears

  • TOOLS

    • A one stop shop for A.I. Content Creation

  • FEATURED CREATOR

    • Curious Refuge is at it again

  • FEATURED EVENT

    • The first full length A.I Parody Film premieres this week in Los Angeles

    • Framework Overview: Utilizes a two-stage process involving Frames Encoding and a Diffusion Process to animate portraits with expressive facial expressions and varied head poses.

    • Diverse Applications: Demonstrates versatility across singing, talking, and performances in multiple languages, supporting a wide array of portrait styles from historical figures to contemporary digital creations.

    • Technological Innovation: Employs advanced mechanisms like Reference-Attention and Audio-Attention, alongside Temporal Modules, to ensure the preservation of character identity and realistic motion dynamics.

    The EMO project marks a significant leap in digital media, offering novel possibilities for content creation by merging vocal audio with static portraits to generate dynamic, expression-rich videos.

  • Pika has announced a new Lip Sync feature for its subscribers, integrating ElevenLabs' AI-generated voices to animate speaking characters in videos, enhancing realism. This feature supports text-to-audio and uploaded tracks, offering customization in voice styles.

  • Exclusive to Pika Pro users and "Super Collaborators," it represents a significant step towards removing barriers to AI-driven narrative filmmaking.

  • Meanwhile, concerns about AI video training data persist in the community, highlighting the ongoing dialogue around ethical AI development.

NEWS: Your Weekly Tech Update

  • Adobe introduces a new GenAI Music Tool

    • Adobe's Project Music GenAI Control, revealed at the Hot Pod Summit, is a prototype tool enabling users to generate and edit music through text prompts, without needing professional audio skills. This innovative tool allows adjustments in patterns, tempo, intensity, and structure, and can remix sections or create loops. Developed with the University of California and Carnegie Mellon University, it's an early-stage experiment aiming to integrate deep audio editing capabilities akin to Photoshop's control over visuals. Further details are pending on its public release and capabilities.

  • Google has more bad press

    • Google apologized for the inappropriate and inaccurate images generated by its AI tool, Gemini, which produced ahistorical and racially diverse images for specific historical contexts. Google acknowledged the issue, emphasizing the challenge of balancing diversity and historical accuracy. The company has paused Gemini's people-generating feature to make improvements. This incident underscores the broader challenges and criticisms Google faces in its AI development strategy, highlighting the need for careful consideration of ethics and diversity in AI.

  • Microsoft partners with Mistral

    • Mistral, partnering with Microsoft, introduces "Mistral Large," a multilingual text generation AI for enterprises, excelling in complex reasoning tasks. This model, notable for its extensive language support and nuanced understanding, ranks highly on the MMLU benchmark, trailing only GPT-4. Available via API and Azure AI, this partnership also brings a $16 million investment and Azure distribution to Mistral, expanding access to their technology. Additionally, Mistral launched an optimized "Mistral Small" and a chat app, further broadening its innovative offerings in the AI space.

  • Meta making strategic partnerships

    • LG and Meta have announced a partnership to enhance their XR (extended reality) ventures. This collaboration aims to combine Meta's platform with LG's content and service capabilities, especially from its TV business, to create a unique ecosystem in the XR domain. This partnership also promises significant synergies in developing next-generation XR devices, potentially including a new Quest Pro model to rival Apple's Vision Pro. LG, with its extensive experience in consumer electronics, is expected to focus on manufacturing, leveraging its content and services to enrich Meta's Quest headsets.

  • Apple is shifting gears

    • Tim Cook announced Apple's commitment to making significant advancements in Generative AI (GenAI) during the company's annual shareholder meeting. This shift in focus comes as Apple reallocates resources from its electric vehicle project to GenAI initiatives. Despite being slower to adopt GenAI compared to its competitors, Apple plans to enhance Siri, Spotlight, and other services with GenAI capabilities, aiming to improve user interaction and automate tasks like presentation and playlist creation. This strategic pivot is underscored by Apple's increased publication of GenAI-related research and the development of new models and tools

TOOLS: Give Yourself Powers

  • A one stop shop for A.I. Content Creation

  • Curious Refuge does it again. One of the top A.I. Film School teams has created another concept parody film.

  • Just imagine how you can take scripts and books you are trying to make into feature films and create a pre visualization in a week. The tools are there to enhance the creatives.

Thanks for reading.

See you next week!

p.s. if you want to sign up for this newsletter or share it with a friend or colleague, you can find us here

Thank you for your support!