Memgraph & NULLs: Supporting NULLs In Lists

Aug 15, 2025 by RICHARD 44 views

Supporting Lists Containing NULLs in MGP Parameter API: A Deep Dive

Hey guys, let's dive into a specific issue we're facing with Memgraph's MGP Parameter API – specifically, how it handles lists that might contain NULL values. This is a bit of a head-scratcher, especially when we're trying to keep things compatible with other graph database systems, like Neo4j. We'll explore the problem, why it's happening, and what we can do to make things better. This article aims to offer a detailed look at the challenges and potential solutions, ensuring you have a solid understanding of the situation.

The Core Problem: NULLs in Lists

So, the heart of the matter is this: the mgp::AddFunction in Memgraph isn't super flexible when it comes to lists that can contain NULLs. Right now, we have a way to specify a list using the mgp::Parameter constructor. You can tell it, "Hey, this parameter is a list of anything." But here’s the catch: "anything" in this context doesn't include NULL. This is where things get tricky.

Consider this scenario, which will be familiar to those of you working with graph databases. In Neo4j, you can run a query like this, and it will work perfectly:

RETURN apoc.coll.toSet([1, 2, 3, null])
// := [1, 2, 3, null]

It's pretty straightforward, right? You're creating a list that includes some numbers and a NULL value, and then converting it into a set. Neo4j handles this without a hitch. However, in Memgraph, using the collections.to_set function to achieve the same result leads to an error.

RETURN collections.to_set([1, 2, 3, null]);
// Client received query exception: 'collections.to_set' argument named 'values' at position 0 must be of type LIST OF ANY.

This is because the collections.to_set function, like many functions in Memgraph, isn't designed to handle NULL values within lists. The error message clearly points out that the function expects a list of anything, but not NULL.

This difference in behavior creates a compatibility issue. If you're trying to migrate queries or integrate with systems that rely on NULL values within lists, you'll run into problems. It forces you to either modify your queries or find workarounds, which isn't ideal. The goal is to make Memgraph as flexible and compatible as possible, so we need to address this limitation.

To really drive this point home, let's break it down further. The problem boils down to the mgp::Parameter definition and how it interprets "any". The current implementation doesn't provide a straightforward way to explicitly state, "This list can contain anything, including NULL." This is the crux of the issue, and the focus of the proposed solutions.

Reproducing the Issue

To make sure we’re all on the same page, let’s walk through how you can reproduce this problem. It's really simple, thankfully.

Ensure you have Memgraph 3.4 or a later version installed. This is where the issue has been identified, so you'll need a compatible version to test it. Make sure your Memgraph instance is up and running.
Load the MAGE collections query modules. This is essential because the collections.to_set function, which we'll use to demonstrate the issue, is part of these modules. If you don't have MAGE loaded, you won't be able to run the test query.

Run the following Cypher query:

RETURN collections.to_set([1, 2, 3, null]);

Observe the error. You should see an exception in your client, something like: "'collections.to_set' argument named 'values' at position 0 must be of type LIST OF ANY." This confirms the problem.

That's it! You've now successfully reproduced the issue. By following these steps, you can verify the behavior and understand why lists containing NULL values aren't currently supported as expected. This straightforward process underscores the need for a fix or enhancement to the mgp::Parameter API.

Expected Behavior and Proposed Solutions

Okay, so we know there’s a problem, what should happen instead? Ideally, we want Memgraph to gracefully handle lists that include NULL values. The expected behavior is that the query RETURN collections.to_set([1, 2, 3, null]); should execute without throwing an error, and the result should be a set containing 1, 2, 3, and NULL – just like it would in Neo4j.

But how do we get there? The core of the solution involves modifying the mgp::Parameter API to correctly identify lists that are able to handle NULL values. Here are the primary options that have been put forward:

Introduce a New Parameter Type: AnyIncludingNull: This approach would involve creating a new type specifically designed to represent "any type, including NULL." This is a safe approach, as it wouldn't break any existing functionality, while providing a new type to specify that a list can contain null values. It would be clear and explicit.

The downside here is that it adds to the complexity of the API. Developers will need to learn about and understand the new AnyIncludingNull type, in addition to the existing types. Also, it might not be immediately obvious when to use this new type over Any.
Modify the Existing Any Type: This approach would change the existing Any type to inherently include NULL values. The appeal of this approach is that it might simplify the API, as you wouldn't need to introduce a new type. The idea would be that Any just means anything, including null.

However, this is a risky approach. Many query modules currently rely on the assumption that "any" does not include nulls. If you change Any to include null, then a lot of existing code could break. This approach could introduce unexpected behavior and compatibility issues.
Enhance mgp::Parameter Constructor for Lists: This is the preferred approach. The idea here is to provide a specific constructor for list types. This constructor could be designed to explicitly indicate whether a list can contain NULL values or not. It would give developers more control and flexibility without fundamentally changing the meaning of existing types.

This approach is less risky than modifying the existing Any type, and it’s more explicit than introducing a new type. It would ensure that developers consciously choose whether or not to allow NULL values in a list, making the API more robust and less prone to unexpected behavior.

Ultimately, the best solution is the third option: a new mgp::Parameter constructor designed to handle lists that can include NULL values. This would give developers the flexibility they need while minimizing the potential for breaking existing code. It's a balance of functionality and backwards compatibility. The idea is to make sure that Memgraph is easy to use for everyone, regardless of the specifics of the data.

Conclusion

So, there you have it, guys! We've walked through the problem of supporting lists containing NULL values in Memgraph's MGP Parameter API. We’ve seen the issues it causes and considered some potential solutions. It's a small but important detail when trying to make Memgraph more compatible and user-friendly. The key takeaway is that the proposed change aims to enhance the flexibility of the API. By providing a dedicated mechanism for handling NULL values in lists, we can make Memgraph more robust, flexible, and compatible with a wider range of use cases.

I hope this deep dive has been helpful. Let me know if you have any questions, thoughts, or suggestions. Keep building those awesome graph applications!