Opera launches DeviceTest.ai to assess AI readiness on PCs & Macs

Opera has introduced DeviceTest.ai, a tool designed to determine whether a user's computer can handle on-device artificial intelligence (AI) and large language models (LLMs). This new benchmark will assess the AI readiness of both PCs and Mac computers, signifying an important step in decentralising AI processing from servers to individual devices.

DeviceTest.ai is integrated with the latest version of the Opera Developer browser, the first to support built-in local LLMs. Users can initiate the test by downloading an LLM and running several assessments. These tests evaluate parameters such as tokens per second, first token latency, and model load time. After the evaluation, the tool provides users with a complete overview of their computer's readiness for on-device AI.

The company explained that the introduction of an AI benchmark tool became necessary as no existing tool was accessible to the general public. Opera realised the importance of such a tool during the development of multiple AI solutions for its browsers. Whether one is a casual user, an AI enthusiast with advanced hardware, or a researcher, this benchmark tool aims to cater to everyone’s needs.

To use DeviceTest.ai, users must access the specified link using Opera Developer. After reading the relevant information, they can click 'Run Test', select a profile for the test, and then begin the test. The chosen profile determines the complexity of the LLM used, ranging from basic to highly resource-demanding models. Depending on the hardware, the testing time could range between three to twenty minutes.

Upon completion, the results are presented with a colour-coded system: green indicates AI readiness, yellow denotes AI functionality but with potential limitations, and red signifies that the device is not AI-ready. The three key indicators users should focus on include Tokens Per Second (TPS), First Token Latency (FTL), and Model Load Time (MLT).

Tokens Per Second measures how many average-length words a device can process each second. First, Token Latency evaluates the time required for the model to generate the first word after receiving a prompt. Model Load Time measures how quickly a device can load the LLM into its RAM. These indicators provide insight into a device’s overall performance when handling local AI tasks.

The tool repeats each task multiple times to obtain accurate and reliable averages. Post-evaluation, the benchmarking tool displays the system's specifications and the AI readiness of different tasks based on the LLM chosen. Users can further download their results as a CSV file for detailed analysis or share the results with others via a link provided at the end of the test.

For those concerned about data privacy, Opera ensures that only the test results are processed, with no personal data being collected or associated with specific users or IP addresses. Users are advised to refer to the company's Privacy Statement for more detailed information on data privacy practices.