New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to partition an NC mesh #536
Comments
That's pretty cool! I would like to add a reordering functionality in the NCMesh initialization, but haven't gotten to it yet. I would also appreciate any information on Gecko. |
Thanks for sharing Dan, that's pretty cool :). @acfisher had more experience with Gecko when he integrated it a while ago (see Can you move step 5 earlier in your workflow? If the coarsest mesh is well-ordered, the NCMesh operations should produce reasonable-looking partitionings (you will also not need METIS). |
Parallel non-confoming meshes do not use the partitionings generated by I have a branch where I add more reordering methods, including the current reordering based on Gecko - I can push that work to GitHub if someone is interested to take a look. I have not worked on it for a while, so I don't remember how close it is to a version that can be merged into master. |
Also, this is not obvious to me: is the mesh non-conforming from the beginning, i.e. step 1, or do you perform the non-conforming refinement later? |
Hey,
The mesh can be non-conforming from the beginning as our software can take in a MFEM mesh format which supports non-conforming meshes.
I actually think for this problem moving step 5 earlier would work. I think what Dan is trying to do in steps 2 through 4 is to call METIS to re-order the elements before the NC code just splits the global element sequence into N parts (based on suggestions from issue #258) in case the original mesh is not well-ordered. However, since METIS makes no guarantee about making N equal parts, we end up getting reasonable partitions + sprinkles floating around when the splitting is performed. |
METIS (used by |
I believe in this case METIS is generating contiguous partitions but these partitions are neither completely equal nor spatially next to each other in increasing rank (neither which are guarantees). I think because of this, chopping up the mesh into N parts based on global mesh numbering will result in mostly contiguous partitions that will have additional elements from the next or previous METIS partition which can be spatially located anywhere. I think that's why Dan's partitioning looks like it has some structure but with some sprinkles all over. I think I should be able to show this on a much simple (and smaller case) - let me know if we're still in disagreement about the METIS strategy and I'll put that up on Monday. |
If you are reading-in an AMR MFEM mesh then you have the whole refinement hierarchy and the default partitioning that MFEM will use when transitioning from So obtaining a good partitioning for an AMR MFEM mesh relies on the coarsest starting mesh from which the AMR mesh is derived. In the example above, is the starting mesh (before AMR) uniform, Cartesian, |
In the example above, the starting mesh is uniform, Cartesian, |
Gecko is definitely the better choice for reordering. The METIS approach may work better when applied to the initial Cartesian mesh compared to applying it to the refined AMR mesh. Even with Gecko, it is probably better to apply the reordering to the starting coarse mesh. |
I'm fairly certain that Dan is applying METIS to the initial mesh, re-ordering, creating a parallel mesh, and then adapting. |
@Dan2997925 can you update us on the status of this issue? |
I thought @tzanio and @jakubcerveny would enjoy this partitioning. The next MFEM T-shirt? Although not really optimal from a computational point of view! How can I improve this?
Here is what I am doing:
I'm guessing I need to replace step 2 and 3 with Gecko? Has anyone else used Gecko, or some other space filling curve type reordering?
The text was updated successfully, but these errors were encountered: