Kling as a Name is Not the Greatest of Things. But a Guitar Wielding Panda Sure is!

Edition 34: Add Kling.ai from China to the Sora pile.

Add Kling.ai to the Sora pile.

Source: www.kuaishou.com

Kling.ai is a new text-to-video generation model developed by the Chinese tech company Kuaishou. It's being compared favorably to OpenAI's Sora model, boasting features like creating realistic videos up to two minutes long, simulating real-world physics, and handling complex scenarios with lifelike detail.

Today we’ll focus on Kling. I have a backlog of alpacas and elephants that I’ll send out probably Wednesday.

So let’s dig in to features and functions.

1. Large-scale reasonable exercise

• “KeLing adopts a 3D spatiotemporal joint attention mechanism, which can better model complex spatiotemporal motion and generate video content with larger movements while conforming to the laws of motion.”

  • This mechanism allows Kling to model complex motions in space and time accurately. It can generate video content with realistic, large-scale movements that follow the laws of motion.

2. Video generation up to two minutes long

• Kling AI excels in efficient and scalable video creation. Powered by a robust training infrastructure and optimized inference, it generates compelling footage up to two minutes long with a smooth 30 frames per second frame rate.

Source: www.kuaishou.com

3. Simulating physical world properties

• Utilizing its innovative model architecture and scaling principles, Kling.ai can simulate real-world physics and generate videos that obey the laws of physics.

4. Strong concept combination ability

• Leveraging its grasp of text-video relationships and the potent Diffusion Transformer architecture, Kling.ai can turn users' vivid imaginations into tangible visuals, including fictional scenes that defy reality.

Source: www.kuaishou.com

5. Movie-quality image generation

• Leveraging a self-developed 3D variational autoencoder (VAE), Kling.ai possesses the capability to generate high-definition video at 1080p resolution, enabling vivid and lifelike depictions of grandiose, sweeping landscapes as well as intricate, nuanced close-up details, thereby achieving a cinematic quality akin to feature films.

6. Flexible Video Aspect Ratio Output

• Kling.ai employs a variable resolution training strategy, enabling it to generate the same content with diverse video aspect ratios during the inference process. This versatility allows the outputted video materials to seamlessly accommodate a wide range of intended use cases and scenarios (TikTok, Instagram, etc.)

Source: www.kuaishou.com

The deal

I would consider output in variable aspect ratios “helpful” to say the least, what with the variable social requirements like TikTok, Instgram … They claim two-minute segment length which I am not seeing I saw. Check the X.com links below. I will not pick these apart to play tough guy, but like all others we’ve seen, this is a pretty good looking first date.

I’d love to get ahold of a beta or at least have Paul Trillo compare to his Sora experiences. You can see examples of his work in prior editions here. Rumor has it he swings by time-to-time so maybe he’ll drop something in the comments. ;)

To the uninitiated, Paul was one of a handful of filmmakers asked to take Sora for a spin. Sora is Open AI’s entry into prompt-driven, extended-length, AI-powered video creators.

Source: www.kuaishou.com

Okay - I’ll critique this one. It seems like forced perspective - the more milk that’s poured in, the bigger the glass appears. Based on the size of the pouring hand, to me this is a glass bucket. I’d still drink it.

More Background

Amidst the AI talent exodus and project shutdowns rocking Kuaishou Technology, the TikTok rival seems to be pinning its hopes on this ambitious new AI model called Kling.ai. As the company hemorrhages staff and resources, this cutting-edge language model could be a make-or-break gambit for Kuaishou's future.

Early reports suggest Kling.ai packs a impressive punch, capable of generating video content up to 1080p resolution, with unconstrained aspect ratios and a smooth 30 frames per second – all within a two-minute time limit. For now, Kling is being rolled out to invited beta testers through Kuaishou's Kwaiying app, whetting the appetite of content creators and tech enthusiasts alike.

With the AI race heating up and heavyweights like OpenAI and Anthropic setting a blistering pace, all eyes are on the embattled Kuaishou to see if Kling.ai can be the secret weapon that propels them back into the spotlight. The stakes have never been higher for this once out-performer of China's tech scene.

Additional from testers in the wild on X:

• https://x.com/bdsqlsz/status/1798710076175528354

  • prompt: Traveling by train, viewing all sorts of landscapes through the window.

  • Impressive long cut over two minutes of one of the company demos.

• https://x.com/i/status/1798710829212487943

  • prompt:Little boy riding his bike in the garden through the changing seasons of fall, winter, spring and summer.

  • 1:31 in length. Long cut of another company demo.

Availability

Presently Kling is in Beta.

While there's no guarantee, it is possible Kling.ai could see a wider release sometime in 2024. This is because another Chinese text-to-video model, Vidu, launched in April 2024, so Kling.ai's development might follow a similar timeline. However, this is just speculation, so staying updated with Kuaishou announcements is your best bet for catching the official release news. We covered Vidu here.

I’m sure we’ll have more later in the week. That's all for today. Dodging the slings and arrows of outrageous fortune, Thanks for reading The Ad Stack Edition 33! Feel free to share comments below! Share it around the office - And for the love of God, Smash that heart button!

And thanks for being Stackers each and every one of you.