Resolution order for preloading, in the new execution engine?

[skanev]

I'm trying to optimize the communication with the database and make as few queries as possible. The context can be e bit complicated, so I think I'll manage to explain it best if I lead with an example.

I end up having the following GraphQL query (really, several batched/multiplexed ones, but for simplicity's sake):

query {
  city(id: $cityId) { id name }
  users { nodes { id name city { id name } } }
  cities { nodes { id name } }
}

Resolved in the order written, it would result in three SQLs:

SELECT * FROM cities WHERE id = ?; -- to load the first city
SELECT * FROM cities WHERE id IN (?); -- to preload the city on the users
SELECT * FROM cities ORDER BY id LIMIT 50; -- to load the first page of the connection

Now, if I could choose in what order the to run the resolvers, there are two things that can be done here.

1. If Query.users get resolved before Query.city, then the User.city and the Query.city could leverage batching to run only one WHERE id IN (?), reducing the count of queries to 2.
2. If Query.cities gets resolved first, there is a chance that the cities we need for the other two resolvers are in the first page and we can just use them, collapsing everything to a single query to cities.

I've gotten that to work with the "old" execution engine, but it's kinda messy and needs to rely on private stuff. As far as I can tell, if there is a hook in the new breath-first execution engine that would allow me to pick/override the order in which the query is resolved (on a per-query basis), that could end up being an elegant solution.

I guess the questions are (1) is there a way I could do it and (2) is there some other approach I am missing to accomplish the same result?

Thanks :)

[rmosolgo]

Hey, great question. The main thing that came to mind here was Lookahead (https://graphql-ruby.org/queries/lookahead.html). You could determine whether users' cities were going to be loaded with something like

def self.city(context, id:)
  root_lookahead = context.query.lookahead 
  if root_lookahead.selects?(:cities) 
   # ... ? 
  elsif root_lookahead.selection(:users).selection(:nodes).selects?(:city) ||
    root_lookahead.selection(:users).selection(:edges).selection(:node).selects?(:city)
    # ... ?
  end 

But I'm honestly not sure how to get Dataloader (are you using that?) to order these queries in the right way.

Could you share some details about how you accomplished this before? Maybe that will help me brainstorm 😅

[skanev]

I'm using Dataloader, yes. I haven't got it to work particularly well or particularly shippable. I did two different things to get to one query in this case.

The first one was to identify the connections that will get selected before the query starts resolving by looking at the lookahead, run them preemptively and store the results. I have a whole mechanism where a I have a [model_type, id] => active-record-instance cache in the context that is shared between a RecordFinder, and some custom stuff around ActiveRecord::Associations::Preloader that avoids running SQL for by-id look ups in case the object is already loaded. Once the fields start resolving, the DataLoader would find them in the cache, and the connection queries will find them in another cache. I didn't like this approach in particular, as it works outside of the resolution.

The second one was to ask Claude to figure it out with a DataLoader, and it did, but it was very messy and I didn't quite grok the code. It basically created another dataloader that was there to just delay things, wired it to everything, and it was fulfilling the resolutions only partially so it can sequence them. It ended calling context.dataloader.yield a bunch of times as well, which was messy. I could dig up the code if you want, but it was a slop :)

Both lead me to think that the breadth-first engine is more suited for this (it also makes my custom ActiveRecord::Association::Preloader stuff unnecessary – I'm annotating all the fields that are associations (e.g. I write association :city, Types::CityType instead of field, and then in node() and in connections, I use the lookahead to preload all associations eagerly.

In terms of how it could be implemented, here's a suggestion. At every point, the breadth-first engine has a number of fields that are immediately pending, and has to choose one to resolve. For example, it starts with Query.city, Query.users and Query.citites. There are also a few further-down-the-line like User.city.

If instead of the engine picking, there could be a hook that delegates to the user, it seems to me that the code would be just two rules.

1. If there are pending connections, prefer them over everything else. This allows for building a cache and warming it with the results of the connection. This would solve it with the query I provided.
2. If there are no connections, prefer fields of a type that is not also anywhere in the further-down-the-line fields. If the query also has query { user(id: $userId} { city { id name } } }, this would prefer Query.user, which then queues a User.city. We're then left with Query.city and User.city which ideally could be grabbed together with a DataLoader.

That is ultimately what I told Claude to do when it ended up doing weird DataLoader stuff.

I get that this has some interaction with the suspended fibers of the DataLoader that I haven't quite thought through, but hopefully it's useful.

[rmosolgo]

another dataloader [Source] that was there to just delay things

This idea came to me during lunch today, too -- you could create PrioritySource with something like

def fetch(priorities)
  # Intentionally don't fulfill the others
  return [priorities.min, true] 
end 

(I'm not sure if that works -- maybe it'd actually have to dataloader.yield like you're describing.) Then you'd have to update every field to use that. Yuck 🤷

At every point, the breadth-first engine has a number of fields that are immediately pending, and has to choose one to resolve. [T]here could be a hook that delegates to the user

This seems possible ... When Execution::Next encounters a selection set, it creates FieldResolveSteps for each field, then passes them along to the Runner to actually run later:

https://github.com/rmosolgo/graphql-ruby/blob/master/lib/graphql/execution/selections_step.rb#L89-L91

Implementation-wise, the Runner actually passes those steps to the dataloader:

graphql-ruby/lib/graphql/execution/runner.rb

Lines 52 to 54 in 82afc26

	def add_step(step)
	@dataloader.append_job(step)
	end

The default Dataloader (NullDataloader) simply .calls the steps immediately.

Implementation-wise I can think of two possibilities:

• Extend Dataloader to support re-ordering of pending work. Currently its simply an Array used as a queue:

  graphql-ruby/lib/graphql/dataloader.rb

  Lines 145 to 151 in a538df5

  	# @api private Nothing to see here
  	def append_job(callable = nil, &job)
  	# Given a block, queue it up to be worked through when `#run` is called.
  	# (If the dataloader is already running, then a Fiber will pick this up later.)
  	@pending_jobs.push(callable || job)
  	nil
  	end

  graphql-ruby/lib/graphql/dataloader.rb

  Lines 359 to 361 in a538df5

  	while job = @pending_jobs.shift
  	job.call
  	end

  • Or, reorder the array of continue_steps before they're passed up to runner.add_step. This could actually happen based on some field configuration -- continue_steps.sort_by { |step| step.field_definition.priority }

    But this idea is only half-baked. A FieldResolveStep doesn't have .field_definition populated at this stage -- could it? Also, I'm not sure enqueuing them in a different order would achieve the desired result because they'd still be executed roughly concurrently. The first job would start, then pause, then the second job would start, then pause, and we're back where we started

  • If that won't work (I don't think it will), you'd have to build a dependency system into the steps themselves. There actually is a case where steps explicitly wait on other steps before they proceed to run:

    graphql-ruby/lib/graphql/execution/field_resolve_step.rb

    Lines 485 to 489 in a538df5

    	if @pending_steps.nil? || @pending_steps.size == 0
    	enqueue_next_steps
    	else
    	# Do nothing -- it will enqueue itself later
    	end

    That's how a field waits for its arguments to be loaded, authorized, etc. The argument-related step removes itself when its done:

    graphql-ruby/lib/graphql/execution/load_argument_step.rb

    Lines 94 to 98 in a538df5

    	field_pending_steps = @field_resolve_step.pending_steps
    	field_pending_steps.delete(self)
    	if @field_resolve_step.arguments && field_pending_steps.size == 0 # rubocop:disable Development/ContextIsPassedCop
    	@field_resolve_step.runner.add_step(@field_resolve_step)
    	end

    I don't have a clear vision for how that could be extended to support field-to-field dependencies, but it seems like a clue.

That's what I've got at the moment!