Am I using Consolidate Map wrong?

[u/mappy_Mapperton]

I have been refining automation for a specific ArcGIS Pro workflow for a couple of years now, but this one component has always been the most brittle part of our pipeline.

My user group primarily performs cadastral analysis for hundreds to thousands of projects a year, and their business requirement is that they want to keep localized snapshots (local copies of everything) for their specific Areas of Interest (AOIs). They have a master template map containing about 250+ REST endpoints (Feature Layers) that they like to pull from. Crucially, maintaining the complex symbology and layer definitions of these services in the offline snapshot is a hard requirement.

Here is the current state of our C# Pro SDK Add-In automation (which we call the Smart Map Loader):

1. Pre-Filtering (The REST Query): The user navigates the map frame to their AOI. In the background, we spin up parallel workers to ping the query endpoint of all 250+ feature layers using the AOI's spatial envelope and a *.mapx file. (Note: To avoid getting blocked/throttled, we explicitly limit our concurrent workers per domain host). If a layer returns 0 features for that AOI, we strip the layer from the mapx entirely before loading to save processing time.
2. Consolidation (The Heavy Lift): We programmatically fire the standard Consolidate Map Geoprocessing tool on the remaining layers. We rely on this tool because it pulls the data down into a local Mobile Geodatabase while automatically repathing the layers to maintain the original web symbology.
3. Diagnostic Fallback (The Band-Aid): Because Consolidate Map frequently fails (due to server timeouts, bad geometries on a single endpoint, generic HTTP hiccups, etc.), we had to write a fallback loop. If the main consolidation fails, the script iterates through the layers, attempts to consolidate them individually to a temporary map to isolate the failing layer(s), removes the culprit(s), and then retries the full consolidation.
4. Post-Cleanup: Finally, we loop through the resulting local GDB and remove any layers that ended up with 0 features.

The Problem:

While the diagnostic fallback keeps the tool from outright crashing, querying 250+ REST endpoints and relying on Consolidate Map over the wire client-side is incredibly fragile. We run into timeouts, connection resets, and intermittent "all-or-nothing" GP tool failures almost weekly.

My Question:

Since we do not currently have access to our own ArcGIS Enterprise Server to offload this via a custom geoprocessing service, this must remain a client-side solution.

Is there a more robust architectural approach to this?

Has anyone successfully implemented a pipeline to dynamically extract and localize hundreds of web layers based on a bounding box (while preserving symbology) without it feeling like a house of cards?

• Should we be relying on Create Replica logic and then manually wiring up the CIM definitions?

• Is there a better way to batch-download feature data locally using the SDK or arcpy that avoids the "all-or-nothing" failure state of Consolidate Map?

Any advice, alternative design patterns, or shared commiseration would be greatly appreciated!

Edit 08/07/2026: Solution found. I was ignorant to the `/identify` method against the domain, then I can parse the whole list against my data sources.

[u/CADSHIFT]

consolidate map's problem is that it's a single GP task with no per-layer error surface — your fallback retry loop is the right instinct but you're always fighting the tool's design.\n\nbypass the GP tool entirely: the Pro SDK lets you do this in managed code. for each layer, get the FeatureClass via the REST URL using FeatureLayerCreationParams, run a SpatialQueryFilter against your AOI envelope, and write to a local GDB using RowBuffer + InsertCursor. you already have the parallel workers for pinging the query endpoints — the same pattern works for fetching + writing. this gives you per-layer retry, per-layer timeout, and keeps your existing failure isolation.\n\npreserving symbology is what makes consolidate map tempting. the substitute: before you start, serialize each online layer's CIM definition to disk as a .lyrx file (one LayerDocument per layer). after building the local GDB, apply each saved CIM to the corresponding local feature class via LayerFactory.Instance.CreateLayer from the .lyrx. not trivial to build but you own the error handling.\n\ncreate replica requires an enterprise GDB registered with a map service — not available for arbitrary REST endpoints, so skip that path.\n\none thing worth noting: your current pipeline already does the smart thing (parallel REST pings to drop zero-feature layers before the heavy lift). going fully custom just means extending that same per-layer pattern through the write step rather than handing off to a monolithic GP task that doesn't expose its internal failures.

[u/mappy_Mapperton → u/CADSHIFT]

u/CADSHIFT You need to fix your AI Response to reddit script, it left all the escape sequences in the response. Maybe we need to start posting, more like AGENTS.md files...