Google Gemini: Everything you need to know about the generative AI models

Google is introducing Gemini, its flagship suite of generative AI models, apps, and services, in an attempt to make a big impact in the field. But what exactly is Gemini? How can it be utilized? And how does it compare to other generative AI tools like OpenAI’s ChatGPT, Meta’s Llama, and Microsoft’s Copilot?

To help you stay informed about the latest Gemini developments, we have created a comprehensive guide that will be continuously updated as new Gemini models, features, and news about Google’s plans for Gemini are revealed.

Gemini is Google’s advanced generative AI model family, developed by DeepMind and Google Research. It includes four variations:

– Gemini Ultra, a very large model
– Gemini Pro, a large model with the latest version being Gemini 2.0 Pro Experimental
– Gemini Flash, a faster and distilled version of Pro
– Gemini Nano, consisting of two smaller models: Nano-1 and Nano-2 for offline use

Unlike other models, Gemini is natively multimodal, capable of analyzing more than just text. It has been trained on various data sources including audio, images, videos, codebases, and text in multiple languages.

The Gemini apps, separate from the models, act as interfaces connecting users to the Gemini models. Available on the web and mobile devices, these apps allow users to interact with the generative AI through voice commands, text, and images.

For advanced features, users can subscribe to the Google One AI Premium Plan, granting access to Gemini in Google Workspace apps and enabling Gemini Advanced capabilities. This includes features like running Python code, an expanded context window, and research brief generation.

Gemini Advanced users also benefit from unique features such as memory recall for past conversations, access to Google’s Deep Research feature, increased usage for NotebookLM, and trip planning in Google Search.

Corporate customers can access Gemini through business and enterprise plans, offering additional features like meeting note-taking and document classification.

Gemini is integrated into various Google services like Gmail, Docs, and Slides, enhancing productivity and creativity across platforms. Gemini in Google Sheets organizes data by creating tables and formulas. It can also summarize reviews in Maps, files in Drive, and translate captions in Meet.

Gemini has expanded to various Google products, including Chrome, Firebase, and Google Photos. It also powers Code Assist for developers and security products like Threat Intelligence.

Gemini Advanced users can create Gems, custom chatbots, and access Gemini Live for voice chats. Imagen 3 allows users to generate artwork, while Gemini for teens offers a tailored experience for students.

In smart home devices like Nest cameras and thermostats, Gemini enhances Google Assistant’s capabilities. It can provide AI descriptions for camera footage and recommend automations based on real-time events.

Overall, Gemini models are multimodal and can transcribe speech, caption images and videos, and provide natural language responses in various Google products. Many of the capabilities mentioned earlier have already been developed into products by Google, with more promised in the near future. However, there have been some instances where Google has not fully delivered on its promises, such as with the original Bard launch and the aspirational video showcasing Gemini’s capabilities.

Google has yet to address some of the underlying issues with generative AI technology, such as biases and hallucinations. Despite this, Google claims that its Gemini models, including Ultra, Pro, Flash, and Nano, have various capabilities that can be useful in tasks such as homework, coding, reasoning, and data extraction.

Gemini Ultra, although not currently visible in the Gemini app, is said to have multimodal capabilities that can be applied to tasks like identifying scientific papers and generating images. Gemini Pro, on the other hand, is touted as the best model for coding performance and complex prompts, with the ability to reason across large amounts of data.

Gemini Flash is described as Google’s AI model for the agentic era, capable of generating images, audio, and text and outperforming previous models on benchmarks measuring coding and image analysis. Gemini Nano, a smaller version of the Pro and Ultra models, can run directly on devices like the Pixel and Samsung Galaxy phones, powering features such as Summarize in Recorder and Smart Reply in Gboard.

Overall, Google’s Gemini models offer a range of capabilities that can be useful in various tasks, but potential users should keep in mind the limitations and challenges associated with generative AI technology. Users can receive summaries without a signal or Wi-Fi connection, with a focus on privacy as no data leaves their phone. Nano technology is integrated into Gboard, Google’s keyboard, to power Smart Reply in messaging apps like WhatsApp. In Google Messages, Nano enables Magic Compose to craft messages in various styles. Future Android versions will use Nano to alert users of potential scams during calls. The new weather app on Pixel phones utilizes Gemini Nano for tailored weather reports. TalkBack, Google’s accessibility service, uses Nano to create descriptions of objects for low-vision and blind users.

Gemini models are available through Google’s Gemini API with free options but also offer pay-as-you-go pricing. Tokens are used for input and output, with pricing based on the number of tokens processed. Gemini 2.0 Pro pricing has not been announced yet, and Nano is currently in early access.

Project Astra by Google DeepMind aims to create AI-powered apps and agents for real-time understanding of multimodal data. A small number of testers have access to Project Astra app, with plans to integrate it into smart glasses. However, Project Astra is still in the development phase and not yet a product.

Apple is considering using Gemini and other third-party models in its Apple Intelligence suite, with plans to collaborate with Gemini in the future. Craig Federighi confirmed Apple’s intentions to work with models like Gemini but did not provide further details.

This post was originally published on February 16, 2024, and is regularly updated.