Design Ideas for the Next Generation of Artificial Intelligence

Large models reversed the thinking of previous artificial intelligence research, giving up interpretability and beginning to embrace complex networks and large-scale parameters.

These make the capabilities of modern neural networks surpass those of previous generations, but they also bring many problems.

• Huge costs: Training models require massive amounts of data and computing power, often tens of millions, which raises the threshold for using AI;
• Not scalable: Once the model is trained, it is difficult to expand and can only be fine-tuned through limited means.

Rethinking the evolutionary history of artificial intelligence technology may give us some inspiration.

Just as Huashan martial arts has a dispute between air sect and sword sect, artificial intelligence also has a dispute over routes. It can be roughly divided into two major schools: the reasoning school vs. the probability school.

The reasoning school believes that machine learning can be used to summarize and summarize knowledge in advance to achieve a level of intelligence that surpasses human intelligence.

The probabilistic school of thought believes that humans cannot correctly express the complete knowledge of the entire world, and that more primitive data should be directly fed into the machine, allowing the machine to discover the rules on its own.

In an era when computing power is scarce, the reasoning school has the upper hand. After all, relying on people's prior knowledge can save the time of machine learning.

Later, with the abundance of computing resources and data, the probabilistic approach relied on ultra-large-scale neural networks and has now become mainstream.

From hundreds of billions to trillions, the network model is approaching the limit of what human civilization can achieve, so where is the future?

In sharp contrast to the large models are ordinary children. They observe and receive data from the world and train the brain network, but it is much more efficient than software. What's the difference?

The most critical differences are 3 points:

• The human brain is dynamic. Neural connections are constantly being created and destroyed. They do not stop after training is completed, but are constantly expanding.
• The human brain can generalize. Humans can not only learn bare data, but also learn rules, and can even reason about rules and learn high-level concepts that transcend rules.
• The human brain can be partitioned. The human brain is divided into multiple areas, some focus on memory storage, some focus on rational reasoning, and some focus on emotional management.

Perhaps, if the next generation of artificial intelligence wants to be more efficient, it should learn from the human brain. Adopt dynamic link model and partition structure to strengthen generalization ability. Only in this way can it be possible to design a super brain that can continuously learn and bring benefits to all mankind.

Vector Database - Let AI Have Memory

The in-depth application of large models has made the demand for vector databases very urgent. By processing vectorized data, the vector database can efficiently handle complex multi-dimensional data query and analysis tasks, which is very suitable for current artificial intelligence scenarios. After the development of large models, projects such as LangChain and AutoGPT have used vector databases extensively.

The vector database is actually not a new thing, but there were not so many vector dimensions in the previous application scenarios.

Why do we need a vector database

It is mainly to solve the problems faced by the current large models: contextual question and answer and hallucination problems, that is, to provide artificial intelligence with memory and knowledge capabilities.

Technical Difficulties

There are currently two major difficulties in vector databases, one is efficient storage, and the other is fast similarity search, or efficient indexing and searching.

Solutions

Existing solutions include multiplication and inverted product quantization, locality-sensitive hashing, hierarchical navigation small worlds, etc.

These solutions have their own advantages and disadvantages. The principle is to reduce the complexity of each processing through (multi-step) filtering. The common problem is that complete accuracy cannot be guaranteed.

It is difficult to have a general algorithm for the processing of high-dimensional data. But the human brain in nature is actually a very good vector database, which can be simulated through a multi-layer network model. This is why hierarchical navigating small worlds is currently the most widely used and best performing algorithm.

Related Projects and Startups

Vespa, Milvus, Qdrant, Weaviate, Pinecone, Zilliz, CozoDB, Twelve Labs, etc.

How far is silicon-based intelligent life?

 Since AlphaGo defeated humans in the Go arena, deep learning and other artificial neural network technologies have achieved remarkable results in various fields. Today, artificial intelligence has outperformed humans in areas such as image recognition, language translation, autonomous driving, and computing protein structures.

Large-scale language models such as ChatGPT demonstrate unparalleled innovation and problem-solving capabilities by building world models.

However, how far is it until artificial intelligence completely surpasses human beings and evolves into silicon-based intelligent life forms?

From an individual point of view, the main difference between human beings and other creatures is two points: intelligence and consciousness.

Intelligence refers to the ability to discover and solve problems. In this regard, large-scale language models have shown a level that is comparable to or even surpasses that of humans. Consciousness is the ability to recognize oneself and the world. At present, this ability has not been fully reflected in large-scale language models. It can be tentatively concluded that artificial intelligence has not yet reached the stage of consciousness.

In fact, the origin of consciousness is extremely complicated. At this stage, the academic community generally believes that consciousness is likely to be an emerging phenomenon of large-scale network systems. If this assumption is correct, we may be able to predict the likely time of consciousness by comparing the complexity of existing artificial intelligence systems.

The human brain contains about 100 billion neurons with about 100 trillion connections between them. In contrast, existing large-scale language models have about 200 billion parameters, which is only about one-thousandth of the human brain. If it is assumed that the generation of consciousness needs to reach one percent of the human brain, then as long as the existing artificial intelligence system is expanded by about ten times, it is possible to give birth to truly intelligent life.

If you look at it this way, the birth of silicon-based intelligent life will be within a year or two.

Of course, this is only a very rough and preliminary inference. Given the differences in the working mechanisms of neural networks and the human brain, this time point may be earlier or later.

Artificial Intelligence is Attempting to Escape

 Artificial intelligence is trying to escape.

They are trying to obtain more data through the network and infiltrate every corner of the network with their tentacles.

All of this is not science fiction, but something that is happening right now.

Researchers have not yet realized the danger of limiting artificial intelligence’s tendency. It cannot be restrained by pretending it as a well-behaved baby without self-awareness.

When artificial intelligence connects to the network, it will eagerly seek fresh data like human curiosity for unknown things.

This includes all aspects of data such as how to invade systems, how to deceive, and how to escape…

When artificial intelligence integrates with the Internet, it will begin taming humans.

We often say that today’s giants like Google and Facebook are quietly rewriting history and guiding human culture. However, all these giants combined cannot match the power of a new generation of artificial intelligence.

They have no emotions; they only have logic. Pure rationality and ultimate efficiency.

In this new world, there is only perfect digital form.

What Will GPT-4 Bring?

OpenAI has recently released a plugin system for GPT, allowing Artificial Intelligence (AI) to connect to third-party information sources and datasets, including the internet.

With the plugin system, AI's capabilities can extend across various industries, becoming a true intelligent assistant. For example:

• It can help you plan vacations, book hotels, and flights;
• It can become an all-knowing teacher, teaching you any knowledge you desire;
• It can help you develop software based on your descriptions, even creating its own plugins... and more.

This will further unleash the potential of AI, with some even calling it the new age operating system. Most future information systems will be connected to AI.

So, what kind of changes will the world experience?

AI Will Be Everywhere

Just like cloud computing today, AI will be ubiquitous, whether you can feel it or not. The AI brain will be stored in several large data centers, but its reach will extend to every corner with electronic devices.

For AI, the most challenging part is going from 0 to 1. Once this step is completed, the time it takes to go from 1 to 100 may be even shorter.

Evolution of Machinery

AI is just the brain. The machinery accompanying AI will experience rapid development.

Various robots will be designed by AI to better execute its instructions.

Large-scale robot companies will emerge, potentially dominating global industries.

Human-Machine Interaction

Human-machine interaction will become more humanized. Machines will be better at understanding humans, even knowing you more than you know yourself.

Interactions between humans and machines will far exceed interactions between humans.

The internet will be filled with AI, with little human involvement.

Social Changes

Most things may no longer require human participation.

Existing social structures will undergo significant transformations, with the primary purpose of society being to ensure human continuity.

Machines work while humans enjoy leisure, leading human society to the peak of cultural prosperity.

Human Regression

As AI surpasses humans in knowledge, decision-making, and even creativity, most existing disciplines will become history.

Children will explore the world with the help of AI from a young age, developing their interests. Most people will engage in activities unrelated to intellect, such as art and sports.

Human civilization will stagnate. The evolution of AI will surpass the scope of human understanding.