Context Engineering: Connecting the Dots with Graphs — Stephen Chin, Neo4j

Hello everybody and welcome to my session at a engineer code summit and I'm going to talk a bit about how you can connect the dots with graph technology and solve problems like

context engineering um improving retrieval patterns and also agentic memory. So we're going to have a lot of

memory. So we're going to have a lot of fun. My name is Stephen Chin. I'm VP of

fun. My name is Stephen Chin. I'm VP of developer relations at Neo Forj and you can find me at all the different social media outlets with my handle Steve on Java. So excited you're all here to join

Java. So excited you're all here to join for the session today. And I think this is how a lot of us have felt the past couple years as AI technology has

basically taken our jobs away. We've

become slaves to um AI programming, to prompt development, to building things off AI models. Now, it's not all bad. I

mean, we we have a lot more time to to play games, to hang out in the matrix, but what we really want to be doing is we want to be doing things which are higher value. So this is where context

higher value. So this is where context engineering comes in the picture and transforms what we've traditionally been doing with kind of oneshot clever phrasing prompt engineering to get

different results out of the AI and we're evolving that to have a more dynamic and and wider scope of things which we're feeding the AI as context

which gives us much better results. So

um this allows us to feed the desire of agents to get even more context and information to do things together. Um to

have more dynamic models and applications to make our applications goal driven. Um selectively curate the

goal driven. Um selectively curate the information for the relevancy of the particular domain which we're working in. So if you're working in a um an

in. So if you're working in a um an enterprise domain, if you're working with a lot of business context, this is particularly important. And then we can

particularly important. And then we can structure the input and get a lot more signal over all the noise of what's being entered into the model which is one of the biggest problems with models

today. Um huge context windows but very

today. Um huge context windows but very little attention focus and simply not looking at the right parts of the context to give us good results. And

this allows us to think not like prompt engineers but like information architects where we're building the model context which actually gives us

superior results coming out of the AI.

And this evolves us from being um you know your traditional trapped to matrix worker to being superheroes. So this is this is where we want to be. We want to be in control of our destiny. We want to

be able to give the agents all of the information all the context which they need to perform the task and to do exactly what we want to get for results out of it. And there's a lot of tools at our disposal now which allow us to

manipulate, control the context and really um feed the AIS all the information which they need to be successful. So in the kind of the scope

successful. So in the kind of the scope of context engineering there's a whole bunch of things which are clearly um part of the domain that

this now encompasses. So one is prompt engineering. Of course, we need to

engineering. Of course, we need to design engineer good good prompts. Um,

make sure that the AI actually has the right instructions, the right information and the right grounding which it needs to do its job well. But

we also need to pull in from different data sources by using things like retrieval, augmented generation. So um

rag is still very relevant for the ability to pull in data from enterprise context from different business contexts and then supply that as additional

information to the AI that it can use to make decisions. Um pulling in state and

make decisions. Um pulling in state and history as well. So now we actually want our models want memory um both short-term memory so they can collaborate with each other and also

long-term memory so they remember the conversation state the history and um they can do more effective long-term operations and we also want to be able to structure

the output in a meaningful way so we can actually feed into not only other applications but other tools and things which we need to collaborate with and integrate our um context with. And when

you put this all together, this is kind of the the scope and domain of context engineering. Now, one of the big focuses

engineering. Now, one of the big focuses of this is all about memory. So, it's

all about how we capture the AI memory and what we're able to do with it. So,

um there's kind of two main categorizations of memory. One is

short-term memory. So, this is what the AI is currently working with on the current tasks. Um, we want to compress

current tasks. Um, we want to compress as much information as possible into the short-term memory and give it relevant results which are high up in the search window. Um, be able to integrate tool

window. Um, be able to integrate tool results into this as well, although you know not give it too much information from tools, especially from previous exchanges where the tools might have dumped a lot of output or information

which will fill our context window. Um,

and in addition to this, we also need to mix in long-term memory. So things which you've learned over a long set of conversations which might be episodic.

Um we need to figure out the semantic and the structural meaning of past conversations. Um kind of pull this out

conversations. Um kind of pull this out into things which can either be used as instructions for the AI and also for procedures and operations which we can

use to to guide and plan the artificial intelligence. And um when we put this

intelligence. And um when we put this together, this helps us pull the more relevant context higher up into the context window, fill in the gaps, and then avoid a lot of the noise, which

gives us bad results or um hallucinations or other problems coming from our AI applications.

And um memory is really the core of what we need to accomplish. um you know if you're plugging yourself up to the matrix this is where you si synergize all of your memories all the things

which you want to get into the AI together with um your own own bind your own neural network that you want to um express and it's extremely important right now because LMS are only as good

as the quality of the response that they're getting from the data. So if you give them bad data if you get them garbage then you are going to get garbage back out again. So, we needed to give them the the right information in

the context window and kind of limit and give it um move it up as high as possible. Be able to do more dynamic

possible. Be able to do more dynamic prompting with things like DSPY and BAML. Um ability to do more reasoning um

BAML. Um ability to do more reasoning um so we can do internal context engineering on top of our data. This

will turn us from human developers into agents where we're actually using more agent technology to fuel our applications to build things. Um, and

then this allows us to focus more on the time which we're doing our tests rather than just focusing on the time which we're training our models. Um, so

together when we have more context now it allows us to do better things but then it's still important that we really have structured information relevant inputs and this improves the reliability

and the explanability of our models that come out of it. Um so one of the ways which we can do this is by leveraging knowledge graphs. Knowledge graphs are a

knowledge graphs. Knowledge graphs are a technology which has been around for a while but they're very applicable for AI because they fill in that gap between

the AI which is very good at um creating things building things kind of pulling from different sources it has but um structured information knowledge graphs are a structured representation to

understand a bit about how a knowledge graph is constructed.

It's typically built with facts which are are nodes about people, places, events or things. Those are linked together by relationships or um um lines

between them which reference how those things are related. It's very easy for both humans and LLMs to read knowledge graphs. So both acts as a um organizing

graphs. So both acts as a um organizing concept, but also a way which you can understand what your AI is doing and actually look at some of the data behind it. And um it can be also very useful as

it. And um it can be also very useful as a digital twin of your organization, of your supply chain, of um a whole bunch of different processes in your

organization. And the basic construct of

organization. And the basic construct of a knowledge graph is um nodes which represent different people in the situation, relationships, and then you can attach properties to these nodes. So

this is an example of a knowledge graph where you have two people, they know each other, they live with each other.

Well, one person lives with the other.

So I guess technically it's Ann's house and they both drive a car. Now the car is owned by Dan or or by an driven by

Dan as well. So they both have a relationship with the car and they have a relationship with living each other.

And you can see the there's attributes on this. How long has Dan lived or

on this. How long has Dan lived or driven the car, the type of car? So it's

a Volvo. Um it's a um model V70. some

information about it and also some embeddings. So we can also encapsulate

embeddings. So we can also encapsulate embeddings on the graph as well. So we

can do vector lookups and this allows us to do fairly complex things as we build larger knowledge graphs to capture all this information. And what knowledge

this information. And what knowledge graphs gives the benefit of is all of that knowledge context and enrichment that we can build into a representation

of knowledge in addition to LLMs which have kind of that language reasoning and creativity and when we put them together we can do really powerful things.

Um so we talked a bit about rag being an essential part of context engineering and a even better way of doing rag is

graph rag. Now what is graph rag? So

graph rag. Now what is graph rag? So

graph rag is any retrieval pipeline which also uses graphs as part of the retrieval process. And so um an example

retrieval process. And so um an example of this is a user asks a question um it goes to the LM and it does a search and it asks for if there's any relevant information which will go as a

query out to a knowledge graph. This

then gets passed in as additional context to the LLM when it's answering the question and then the LM gives an enriched answer which is more relevant.

So it's it's a will give you more relevant results than just a vector similarity search because you also have information about relationships about nodes about community grouping more

context. So you can now get domain

context. So you can now get domain information factual information structured knowledge on your subject.

You can explain what the LM is actually doing because you can see the part of the knowledge graph which got passed to the LM. And you can also evolve the

the LM. And you can also evolve the knowledge graph over time. And you can now start to implement overlays like role-based access. So you can say only

role-based access. So you can say only these people get access for example in a um patient information system only the doctor would have access to the

diagnosis but only the person who um handles the administrative information would have access to phone numbers or addresses or other personal information about the patient. So it allows you to

kind of overload overlay that role based access directly on the knowledge graph and then instruct the LM on what information it's allowed to respond with and um knowledge graphs allow this sort

of explainable AI. So in a in a large graph with a lot of nodes and a lot of information now you can store the learnings from the user and agents at the interactions in the graph context.

You can start to visualize conversation flows with the addition of reasoning.

You can analyze the context data of agent systems about performance um identify opportunities for improvement over time of the um either the um the quality of the results which

you're passing in the relationships um removing duplicate nodes so that you get better quality results coming out of it.

So it gives you a lot of control over the application and the ability to modify and control what the AI is answering kind of like you're you're training in a in a dojo. So I think you

know in the in the film Neo spends a lot of time doing virtual training improving his skills with different programs he's loaded up and um this is how we're able to do a lot of amazing things like this

demo which I'm going to show you. So the

first demo we're going to show is a um graph rack demo using the LLM knowledge graph builder. So I've already set up a

graph builder. So I've already set up a Neo Forj aura instance. This is the um um online free version of Neo Forj. You

can see I have a a running instance with a bunch of relationships loaded up. And

to load up those relationships, I use the knowledge graph builder. The

knowledge graph builder is a very simple web application. It's open source and it

web application. It's open source and it lets you do a couple things. So, it lets you upload files. So, you can drag and drop different files into the user interface. Before the presentation, I

interface. Before the presentation, I loaded up a couple representative files of a um supply chain use case. One is a supply chain document and as you can see

here it has a whole bunch of information about different artifacts um and the digital signatures of them and the relationship of them. And the

second one is the more interesting one.

This is a a VEX document which is a security standard and it talks about some vulnerabilities um in this case inside the Jackson library and talks a bit about um how to

remediate with it, which versions are affected um which commits fix it and all that good stuff. So um we have quite a bit of

good stuff. So um we have quite a bit of information which we loaded up and then what I've done is I've already dropped those into the knowledge graph and we can take a look at what got generated by

the LM. So it takes this through an

the LM. So it takes this through an ingest phase um where the LM actually builds out a knowledge graph and then we can see that some of these nodes

represent different parts of the um VEX document. Um here we can see some

document. Um here we can see some information about um um who found the vulnerability, information about the um vulnerability database URL and um all

this stuff is connected with different relationships and this allows us to query, navigate and traverse this information to build better responses for the LM. So what we're going to do in

this demo is we're going to take this knowledge graph which we built and then we're going to run an LLM which does a two-pass process. The first pass it's

two-pass process. The first pass it's going to do a vector lookup and find a similarity search to find related nodes in the knowledge graph. And the second

pass it's going to take those nodes which are related to the result find related nodes and then pass those in as context to the LM. And ideally what we'd

what we'd like to get from the LM is that um it will answer questions with information it has from the knowledge graph and then it won't be able to answer questions or refuse to answer

questions with things which are outside that knowledge um pool. So let's ask it [gasps and sighs] um about vulnerabilities

in the in the Jasper library. So Jasper

is another um Java library that's very commonly used. It wasn't actually

commonly used. It wasn't actually referenced in the VEX document. So,

we're in this case, we're hoping to get a no response. Okay, so that's amazing.

I I made a typo. I should have said Jackson.

Let's see what we get when we um ask about the Jackson library, even with the typo, because LMS know that humans are imperfect and they're very good at fixing our mistakes. And um here we can see that it actually pulled out

information about the Jackson databind library with an XML injection attack. It

knows a bit about the vulnerability, what version it's in, um whether it's fixed and at which version it's fixed and um all this information is is pulled

from and aggregate off the knowledge graph. So um it gives us quite a lot of

graph. So um it gives us quite a lot of information um very detailed and very focused because it's rounded in a um

data which is um very um complete um it's finite and it's something we can edit modify and change the response over time. So knowledge graphs are a very

time. So knowledge graphs are a very powerful tool. It allows us to do things

powerful tool. It allows us to do things like this where we can um get better responses and better answers. And now

with knowledge graphs at our disposal, now we can we can go and we can fight the um the evil agents. Actually, it's

kind of ironic that the um the agents of the Matrix film were um the bad guys, but actually they operated and acted a lot like modern agents where we're

having LMS collaborate, pull together, and um cooperate on different algorithms. And even the agents in the film had different personalities and um different types um kind of like

individual agents. So um we need to

individual agents. So um we need to power up and and get our um matrix and graph skills up to a level where we can go tackle the agents with new tools like

memory retrieval. So we we talked a bit

memory retrieval. So we we talked a bit about graph retrieval and um graphs are also a great tool and mechanism which you can use to do um memory retrieval as

well. So we can do search

well. So we can do search um in in memory use graph memory retrieval tools. We have an open- source

retrieval tools. We have an open- source MCP server which does a lot of this for graph retrievalss. And now you can query

graph retrievalss. And now you can query the graph not only for knowledge graphs but also vectors as we did in the last um example. And we can also use graph

um example. And we can also use graph data science algorithms like um community groupings or k nearest neighbors or different graph algorithms which allow us to get um pull some

insights out of the relationship and the structure of the graph. Pull back

relevant information and then pass this as additional context um either for short-term or long-term memory into the agent loop. um where now we're feeding

agent loop. um where now we're feeding the agent with additional information and context from either um a short-term memory source about the current conversation or knowledge pulled in like

kind of what we showed in the previous example from a graph or from a long-term structure of memory where we memorize um previous conversations give them

temporal information and then structure those into a memory that can be retrieved from the graph. And um now we're able to use graph memory to capture knowledge in the form of entities and relationships between them where some nodes have the relevant

properties such as text details embeddings time and location on top of them. So this is kind of a visual

them. So this is kind of a visual representation of our of our graph our memory graph. Some of these properties

memory graph. Some of these properties get vector embedded and this enables us to do vector-based semantic search. So

now we can do semantic search on the graphs via projections into vector space. But then we can also use

space. But then we can also use algorithms like K approximate nearest neighbors, community groupings, um page rank algorithms on top of the

graph to answer different types of questions and to kind of bubble up the most relevant results into the context.

This gives us quite a lot of power because we can do both the vector embeddings, but we can also do additional graph algorithms on top of it.

Okay, so now we have our our superpower with our our graph where we're able to do amazing things which aren't possible just with um vector embeddings and similarity searches kind of like like

the bullet time and the Matrix films. Um this will allow us to do amazing stunts and to evade the um um the agents. But

let's give a quick example of how this would actually work in practice. So

let's say my question to the LM was let's update this presentation from the last time I presented with Sid who's my colleague in India. Um so we can now

search this information in the graph and there's two relevant people for this right so it's it's me um VP of Devril it's Sid who's a community manager and

the event and the last time we presented it was at an event called GIDS um which is an event in Bangalore awesome developer conference so um now we have

kind of that temporal relationship with the two people and then an event and we can add to this the the memory record at a particular time of the presentation.

So now we're pulling back information about this presentation at a specific point in time and we can pass this in as context to the LM. So when we ask it to update the presentation we both have the

context of who presented where they presented and the time period in which they presented for the LM to build additional information on top of it. And

this is only possible because we can do this um multi-stage query with graphs.

Graphs excel in use cases where you are able to pull in multiple facts which are related um but don't get pulled back in a single query. If you can do it in a in

a one shot or you can get a a single similarity search um standard vector rag is is fine for those sorts of use cases.

It's where the relationships are two or more where you get the real value from doing um graph rag and graph memory.

And um this allows us to do a whole bunch of different types of graph retrievers. So um we could do explicit

retrievers. So um we could do explicit retrieval queries where we have pre-anned retrieval queries with different entry points and retrieving some context. So this gives us some

some context. So this gives us some great information from the graph but we can do better by doing text decipher. So

fine-tuning the LM with schema generating a query for the question and then we can kind of take this to the next level with a genetic traversal where we iteratively navigating over the graph collecting information until we

answer it and using an appropriate set of tools. And to show an example of this

of tools. And to show an example of this um we're going to use the same knowledge graph which I loaded up again in our demo number two but this time we're

going to query that knowledge graph using clawed code. So what I've done is I've hooked up claude code using the um

Neoraj MCP cipher um MCP server which is an open source extension. You can say new forj cipher mcp server which I've already configured with the database

settings and now when we talk to cloud and we ask it a question it can answer with that additional graph context that it can tell us things. So, I put a few keywords into the MCP server like um

graph and database and we can ask it um what do you know about the Jackson vulnerability uh based on your graph database.

And so now in addition to you know pulling in information from its standard knowledge sources it's going to use the NeoRaj MCP server and then query it to get additional information. And you can

see that it's doing this multiplestep plans um search of the graph. So first

it gets back the schema of the graph so it can understand the relationships and how the graph is structured. Now that it understands the schema of the graph, it can go back and it can make a bunch of queries to get information about the

particular vulnerability. So it's firing

particular vulnerability. So it's firing off a bunch of different cipher queries.

Cipher is the um graph query language for Neo Forj and most graph databases support it. It's also now a standard.

support it. It's also now a standard.

The GQL standard that's ratified by ISO is um basically a subset of Cipher. And

now that it got back information about the vulnerabilities, it's pulling back some of the text chunks to get additional context which are hanging off of those nodes. And this way it can give

us a a very complete answer with as much information and context as possible from the graph. And the the main difference

the graph. And the the main difference between this approach where compared to the previous one is the previous

approach was relatively fast but you the level of detail it gave us on the CV was limited. When we give a an an agent in

limited. When we give a an an agent in this case we're giving the clawed agent the ability to kind of have at it for the graph do traversals get information do more traversals you can see that it

gives us back very detailed information about the vulnerability. So it figured out the CV number, the affected vulnerability, the type of attack, the severity, and a technical description of the attack. So it's a lot more detailed

the attack. So it's a lot more detailed than what we got before. And it tells us specifically what versions we need to upgrade to remediate the attack um and gives us some advisory information as

well about this. So um if we were trying to develop a um vulnerability report or something to kind of explain how we should as an organization um address

this vulnerability um using the sort of agentic multi-step um MCP retrieval approach is quite powerful because you can see that it gives us um the best possible response since it's able to go

back and keep pulling additional information from the knowledge graph that it needs. Okay, so we've seen a few different ways which we can apply knowledge graphs to solve and improve

the context of our AI applications. So

now that we know that we need graphs, we need we need a lot of graphs. And the

best place to find information about graph technology and getting a lot of different use cases for graphs is graph academy. It's a entirely free resource

academy. It's a entirely free resource to learn about um both cipher queries.

It has courses on cypher queries, has courses on graph rag with examples in both Python and TypeScript. Um we have some more advanced um graph G gra courses coming up as well. So um a lot

of information which is all free and very hands-on for you to get started and actually build your first application kind of like the ones which I showed here in the presentation. Um now if we

want even more knowledge kind of a wider span we can then go to nodes AI 2026 which is our free online virtual conference. Um this is following up the

conference. Um this is following up the amazing nodes conference we had last week with um over 13,000 registrants. So

it was a huge event and Nodes AI is all about AI for um an entire day with AI focus sessions. The CFP is open right

focus sessions. The CFP is open right now if you'd like to submit and it's free to attend and watch all the sessions.

And if we want to really get down and you know beat the architect at his own game, then we need a lot of deep research and information. And the best

place for that is graphrag.com which is a community site which we support um where it has all of the latest research on different graph approaches um how-to

guides and conceptual information about how to implement graph rag and just a general resource which will help you to uplevel your ability to apply graphs to

different problem domains um with a whole bunch of of the cutting edge latest research which is coming out. So,

um, exciting stuff. Thank you very much for joining me for the session today.

And I hope you learned a little bit more about how you can use graphs to connect the dots and improve your context engineering for your AI applications.