Examples of Communcations Patterns in Elemental

[JBlaschke]

I am not sure how to convert a local Array into Elemental.DistMatrix. The example in the README (and that I have used in #72 ) use Elemental.gaussian to fill a DistMatrix. However for a realistic workflow, I need to load distributed pieces of data (each local to a rank) and "fill" the DistMatrix locally. I seem grid objects mentioned, so there must be some way to specify which indices map to which ranks (like https://github.com/eth-cscs/ImplicitGlobalGrid.jl), but I cannot find any documentations/examples

Clearly Elemental.gaussian! operates locally for each rank -- e.g. if I do:

rank = MPI.Comm_rank(comm)
A = Elemental.DistMatrix(Float64);
if rank == 0
Elemental.gaussian!(A, 4000, 3200);
end

the program hangs.

Can you point me to the right place?

[andreasnoack]

Take a look at

Elemental.jl/test/lav.jl

Lines 39 to 74 in 8308915

	El.queueUpdate(A, s, s, 11.0)
	if x0 > 1
	El.queueUpdate(A, s, s - 1, -10.0)
	end
	if x0 < n0
	El.queueUpdate(A, s, s + 1, 20.0)
	end
	if x1 > 1
	El.queueUpdate(A, s, s - n0, -30.0)
	end
	if x1 < n1
	El.queueUpdate(A, s, s + n0, 40.0)
	end
	else
	sRel = s - n0*n1
	x0 = ((sRel-1) % n0) + 1
	x1 = div(sRel-1, n0) + 1
	El.queueUpdate(A, s, sRel, -20.0)
	if x0 > 1
	El.queueUpdate(A, s, sRel - 1, -1.0)
	end
	if x0 < n0
	El.queueUpdate(A, s, sRel + 1, -2.0)
	end
	if x1 > 1
	El.queueUpdate(A, s, sRel - n0, -3.0)
	end
	if x1 < n1
	El.queueUpdate(A, s, sRel + n0, 3.0)
	end
	end
	# The dense last column
	El.queueUpdate(A, s, width, floor(-10/height))
	end
	El.processQueues(A)

. This is a distributed sparse matrix but the process is similar in the dense case. You first select all the elements to be distributed from local arrays with queueUpdate and then you process the queries.