Microsoft gets exclusive license to use GPT-3 language model. What does the model mean?
Context: Microsoft recently received an exclusive license to use OpenAI’s GPT-3 (Generative Pre-trained Transformer) language model in its own products and services.
- The model uses deep learning method to create human-like text in real-time.
- The third-generation model built by San Francisco-based AI research company OpenAI is available to developers via the OpenAI application programming interface (API), which can be used to develop applications and services.
GPT-3 and its applications
- GPT-3 has 175 billion machine learning (ML) parameters, over 100 times more than its predecessor.
- The ML parameters are crucial for solving problem as they look into multiple configurations to approach a specific task.
- It also represents the skill and complexity level of the model.
- Advantage quantum system has more than 5000 qubits, and 15-way qubit connectivity.
- Its expanded hybrid solver service can run problems with up to one million variables.
- D-Wave commercialises its first 5000+ qubit quantum computer
- The latest generation has been designed to understand inputs in English and to generate output with minimal interactions or adjustments from a user.
- GPT-3 can write (including long-form generative text), translate, comprehend text, and answer closed book questions, reason common tasks, and code.
IBM plans to build an 1121 qubit system. What does this technology mean?
- GPT-3 is trained on Microsoft cloud’s AI supercomputer with various datasets, which consist of text either posted or uploaded on the internet.
- The internet data includes a version of the Common Crawl dataset, an expanded version of the WebText dataset, two internet-based book databases, and English-language Wikipedia.
- For the model’s contextual learning, it has been trained to predict answer with only a description of the task, without any examples, with just one example of the task, or by providing few examples.
Limitations and possibilities
- According to OpenAI, GPT-3 has the tendency to express incorrect text information confidently, and it can provide reasonable output when given inputs are similar to the ones present in its training data.
- The training data mostly has texts in English, meaning the model is best suited for classifying, searching, summarising, or generating in that language.
- The API can be integrated into a particular product, or used to develop an entirely new application.
- As it provides a general-purpose “text in, text out” interface, users could try it on virtually any English language task.
- OpenAI also expects the model to be used by researchers to better understand the behaviours, capabilities, biases, and constraints of large-scale language models.
1.png)
