Bell Labs Internship on Source-aware Language Models (PhD) - #2096741

Job Description

Framework explanation

Large Language Models (LLM) have changed the landscape of AI use in personal and industrial applications. They present an opportunity to increase automation and improve the workflows of people in many different domains. However, LLMs can often misinterpret queries or training data, and sometimes mislead their users with confident sounding yet erroneous answers. This issue arises in parts from making it difficult to confidently find the origin of the knowledge an LLM answer arose from.

One way in which we might increase the confidence of users in a LLM answer is by determining where the knowledge provided in said answer comes from. For example, the user might learn if a particular answer comes from internal Nokia technical documents, external vendor-specific documents, or non-technical documents to determine the level of confidence they have into the answer, as well as where related information is found. By extension, determining the source of information in this way can also help with protecting the users from misinformation. Furthermore, a source-aware model also enables accountability of information, by giving credit to the sources that contributed most to helping a user find the answer to a particular question.

Your role

In this project, you will learn about specific methods that offer potential avenues for tracking the source of LLM knowledge, and be tasked with refining, implementing, testing and extending those techniques to enable information source tracking in LLMs. This task will require being familiar with the training of LLMs and how information is encoded, as well as implementing new training methods testing the validity of assumptions on source tracking. Furthermore, you can expect to research potential new domains of application that can be used to test and evaluate the benefits of Source-aware LLMs.

Additional information

This is a paid internship.

The time duration is flexible and to be agreed (typically 3-4 months).

The starting time is flexible.

How You Will Contribute And What You Will Learn

• You will learn how to leverage existing LLM techniques that offer potential avenues in designing source-aware LLMs.
• You will explore use cases and dataset that provide suitable training data to track the origin of LLM knowledge.
• You will implement new LLM training methods that capture information on knowledge origin.
• You will evaluate your results and iterate on the training method design to improve source learning.
• Ideally, this project leads to a publication at an academic venue.
  
  

Key Skills And Experience

• Student enrolled in Ph.D. in Computer Science or Engineering
• Strong programming skills in Python
• Some experience with LLMs
• Excellent written and spoken English
• Experience with Explainable AI or training of LLMs is a plus
  
  

About Us

Come create the technology that helps the world act together

Nokia is committed to innovation and technology leadership across mobile, fixed and cloud networks. Your career here will have a positive impact on people’s lives and will help us build the capabilities needed for a more productive, sustainable, and inclusive world.

We challenge ourselves to create an inclusive way of working where we are open to new ideas, empowered to take risks and fearless to bring our authentic selves to work

What we offer

Nokia offers continuous learning opportunities, well-being programs to support you mentally and physically, opportunities to join and get supported by employee resource groups, mentoring programs and highly diverse teams with an inclusive culture where people thrive and are empowered.

Nokia is committed to inclusion and is an equal opportunity employer

Nokia has received the following recognitions for its commitment to inclusion & equality:

• One of the World’s Most Ethical Companies by Ethisphere
• Gender-Equality Index by Bloomberg
• Workplace Pride Global Benchmark
  
  

At Nokia, we act inclusively and respect the uniqueness of people. Nokia’s employment decisions are made regardless of race, color, national or ethnic origin, religion, gender, sexual orientation, gender identity or expression, age, marital status, disability, protected veteran status or other characteristics protected by law.

We are committed to a culture of inclusion built upon our core value of respect.

Join us and be part of a company where you will feel included and empowered to succeed.