AI alignment: teaching tech human language

Artificial intelligence used to provoke images of aliens and robots, now it corrects our spellings, helps us drive more safely and saves us time on everyday tasks. It’s well and truly integrated into our everyday lives and AI systems are only becoming more advanced with every passing day – some even exceeding human-level performance in some areas. 

It’s safe to say that AI is one of the most impactful technologies of our time. For business and tech leaders alike it provides a whole new scope of opportunities to develop their businesses. However, to utilise AI’s full capabilities it’s crucial to ensure that the AI is aligned with human intent. 

We can do this with AI Alignment. 

Understanding AI Alignment

AI alignment is a field of AI safety research that aims to ensure that AI systems achieve their desired outcomes and work properly for humans. It aims to create an end-to-end process where an AI-based system can align its output with desired human preferences. 

Imagine playing darts, or any game for that matter, but not agreeing on what the board looks like or what you get points for. If the designer of an AI system cannot express consistent and clear expectations through feedback, the system won’t know what to learn. 

A great human example of this is trying to explain to someone what a road is. This seems relatively simple until we need to do it and realise that it’s quite difficult. Anyone looking into why AI Alignment is a critical step in business should answer the question where does a road begin? How do we explain the nuances of driveways, private roads, level crossings and dirt paths to cars if we can’t explain it accurately to another human? 

At its core, alignment is about agreeing on expectations. Anyone who has managed a team knows how much communication is required to align a group of people – this is essentially what we need to do with the machines we teach, agree on the communication and instruction we give them and then ultimately what the ‘correct’ outcome looks like. With the emergence of more powerful AI, this alignment exercise will be extended to include algorithms. Human feedback is central to this process, and there is still much work to be done.  

As business leaders, it is imperative that we act now to ensure AI is aligned with human values and intent. 

It’s not easy to teach a machine…

Humans interpret things differently and develop preferences based on their personal perceptions. This makes it incredibly difficult to teach machines how humans think and how to tell your “machine” what high performance really is. In order to function properly, AI products need to learn the language of human preferences. 

Up until now, most AI products have been trained using unsupervised learning where we let algorithms derive how to solve tasks by observing humans. But in an application as complex as driving, developers of autonomous vehicles must ask themselves two questions, how do we want this product to behave? And how do we make it behave that way? Agreeing on desired outcomes that align with human intent is crucial, particularly in a safety-critical environment.

For example, the autonomous vehicle industry has already faced many challenges in putting ML-enabled products on the road. AI has not lived up to consumer expectations in this sector. The problem is that there is no single way to drive – what makes a “good” driver? This is partially due to the complexity of the decision making associated with driving, and partially due to the fact that “programming by example” is so radically different from “programming by code.” 

Just like when communicating with humans, the best way to express your intent is to review examples. It’s the same process with an algorithm, a human will review how it behaves and provide feedback, and from this, the algorithm learns and improves. Human feedback can be used to steer AI products very efficiently by shaping the evolving dataset to reflect the developers’ intentions and user expectations.

Taking an iterative approach

Contrary to common belief, AI alignment is not actually a technology problem, it’s a people problem. Ultimately, the ability of the AI system to learn the right kind of rules comes down to the ability of the product developer or service provider to express what it is that they want the product to do. 

If we don’t figure out a better way to do this, we will see a lot of disappointment in the next few years and it’s going to be very difficult to realise the potential of AI. So, it’s in our collective interest to get this right. If business and technology leaders can collaborate closely on alignment, it will help create better products and in turn benefit humans day to day-to-day lives. 

We live in a fast-changing world, and expectations evolve quickly. If you assemble a large dataset, you must expect it to evolve with the expectations of those who will benefit from it. The challenge developers now face is to shape the data they collect with this evolution in mind. This should in turn inform AI products and their future development. 

Alignment is paving the way and is a necessary step when implementing AI. An iterative approach is the best one to navigate the ongoing challenges, expectations and developments of not just humans but AI itself. Ultimately, leaving us with better, smarter AI products. 

Daniel Langkilde

Daniel Langkilde, CEO and co-founder of Kognic, is an experienced machine-learning expert and leads a team of data scientists, developers and industry experts that provide the leading data platform that empowers the global automotive industry’s Automated Driver Assist and Autonomous Driver systems known as ADAS/AD. Daniel is passionate about the challenge of making machine-learning useful for safety-critical applications such as autonomous driving.

How E-commerce Marketers Can Win Black Friday

Sue Azari • 11th November 2024

As new global eCommerce players expand their influence across both European and US markets, traditional brands are navigating a rapidly shifting landscape. These fast-growing Asian platforms have gained traction by offering ultra-low prices, rapid product turnarounds, heavy investment in paid user acquisition, and leveraging viral social media trends to create demand almost in real-time. This...

Why microgrids are big news

Craig Tropea • 31st October 2024

As the world continues its march towards a greener future, businesses, communities, and individuals alike are all increasingly turning towards renewable energy sources to power their operations. What is most interesting, though, is how many of them are taking the pro-active position of researching, selecting, and implementing their preferred solutions without the assistance of traditional...

Is automation the silver bullet for customer retention?

Carter Busse • 22nd October 2024

CX innovation has accelerated rapidly since 2020, as business and consumer expectations evolved dramatically during the Covid-19 pandemic. Now, finding the best way to engage and respond to customers has become a top business priority and a key business challenge. Not only do customers expect the highest standard, but companies are prioritising superb CX to...

Automated Testing Tools and Their Impact on Software Quality

Natalia Yanchii • 09th October 2024

Test automation refers to using specialized software tools and frameworks to automate the execution of test cases, thereby reducing the time and effort required for manual testing. This approach ensures that automation tests run quickly and consistently, allowing development teams to identify and resolve defects more effectively. Test automation provides greater accuracy by eliminating human...

Custom Software Development

Natalia Yanchii • 04th October 2024

There is a wide performance gap between industry-leading companies and other market players. What helps these top businesses outperform their competitors? McKinsey & Company researchers are confident that these are digital technologies and custom software solutions. Nearly 70% of the top performers develop their proprietary products to differentiate themselves from competitors and drive growth. As...

The Impact of Test Automation on Software Quality

Natalia Yanchii • 04th October 2024

Software systems have become highly complex now, with multiple interconnected components, diverse user interfaces, and business logic. To ensure quality, QA engineers thoroughly test these systems through either automated or manual testing. At Testlum, we met many software development teams who were pressured to deliver new features and updates at a faster pace. The manual...