We recently made significant changes to our ArcSDE database setup by splitting our original database into five databases and renaming most of the feature sets. Right now I have a script that opens all the MXDs on a network drive, searches for ArcSDE layers in each MXD, and changes both the source and name of the feature set being referenced. For the most part it works, but it does not update every layer.
Not sure it is relevant, but the ArcSDE database had feature sets grouped in pseudo-directories (feature classes?) within the database. For the MXDs/Layers where the script worked, those pseudo-directories were no problem which leads me to my main guess.
By opening a number of non-updated MXDs and going through the ArcPy documentation I am getting the sense that the issue has to to with user specific database connections. I did check and the feature set spelling matches the one in the script. Given our policy of allowing everyone to recycle MXDs created by other users there are quite a few MXDs where one or two layers are updated and the rest are left pointing to the old, non-existing SDE geodatabase. For the other MXDs, all the layers were added by users other than myself. I could be wrong, but that is my current guess.
While the script does not include user specific information, I think it is referencing the connections I have setup in ArcGIS. Is there a way to force the script to be user agnostic regarding the original database connection (if that is the issue)? Regarding user database connections, all connection names throughout the organization have the same name, but the user name for the database is unique for each user.
UPDATE: The issue of ArcPy trying to open a non-existent ArcSDE database was solved when I realized it was looking for a feature set that is no longer used and added it to the list of ignored feature sets. Unfortunately it only works on the directory specific script I wrote for users to update directories themselves for MXDs that didn’t update with the more inclusive script involved here.
Adding it to the more inclusive, sub-directory crawl script still does not work for some MXDs. Based on comments by Michael Miles-Stimson regarding ArcGIS Desktop versions, I am now looking into the possibility that earlier MXD versions (say ArcGIS Desktop 9.3) work when using the os.chdir function (which is how the directory only script is structured). Trying to see if I can get that into the glob2 code.
Also, I changed the code itself to clean up the Else…If portion of the TRY section by moving the ignore list to the beginning and only having one attempt to update the ArcSDE sourced feature set.
I am using the Python 2.7 installed by ArcGIS 10.1 SP1. The script is as follows:
# michaels
# 06/09/2015
#
# Adapted from script created by:
# emerickr
# 04/14/2011
#
# This script swaps the old SDE data source for the new one
# in all MXDs (recursively) under a root directory.
#
# Import needed packages
import os
import glob2
import arcpy
# Set SDE variables
SDEPath = "Database Connections"
SDEName = "BaseData.sde"
serverName = "172.16.1.41"
serviceName = "5432"
databaseName = "based"
authType = "DATABASE_AUTH"
# Define path to new SDE connection file
newSDEFile = os.path.join("Database Connections", SDEName)
# Set list of all old dataset names
listOfOldDatasetNames = ["sde.sde.Block_Group_Boundary_Clipped_FC",
"sde.sde.Block_Group_Boundary_Unclipped_FC",
"sde.sde.Blocks_Unclipped_FC",
"sde.sde.CA_Assembly_Districts",
"sde.sde.CA_CountyBoundaries_OL",
"sde.sde.CA_Senate_Districts",
"sde.sde.CA_US_Congressional_Districts",
"sde.sde.County_Boundary",
"sde.sde.County_Boundary_clipped",
"sde.sde.County_Boundary_Clipped",
"sde.sde.Fire_District",
"sde.sde.Place_Boundary_FC",
"sde.sde.Postal_Code_Boundary",
"sde.sde.School_District",
"sde.sde.SF_BayRegion_Cities_clipped",
"sde.sde.SF_BayRegion_Cities_nopockets_clipped",
"sde.sde.SF_BayRegionCityLimits_nopockets_OL",
"sde.sde.SF_BayRegionCityLimits_OL",
"sde.sde.Tract_Boundary_Clipped",
"sde.sde.Tract_Boundary_Unclipped",
"sde.sde.Traffic_Analysis_Zones_1990",
"sde.sde.Traffic_Analysis_Zones_2000",
"sde.sde.Water_District",
"sde.sde.Linear_Water_FC",
"sde.sde.Major_Water_FC",
"sde.sde.Pacific_Ocean",
"sde.sde.Springs",
"sde.sde.Water_Polygon_FC",
"sde.sde.Child_Care_Facilities"]
# Set list of all new dataset names
# IMPORTANT: this list must be the same length as the one above it!!!
# All values in this list must correspond to the their indexed counterpart in
# the list above
listOfNewDatasetNames = ["based.public.adm_census_blockgroup_clipped",
"based.public.adm_census_blockgroup_unclipped",
"based.public.adm_census_block_unclipped",
"based.public.adm_ca_assembly_districts",
"based.public.adm_county_boundary_ol",
"based.public.adm_ca_senate_districts",
"based.public.adm_us_congressional_districts",
"based.public.adm_county_boundary",
"based.public.adm_county_boundary_clipped",
"based.public.adm_county_boundary_clipped",
"based.public.adm_district_fire",
"based.public.adm_place_boundary",
"based.public.adm_postal_code_boundary",
"based.public.adm_district_school",
"based.public.adm_region_cities_clipped",
"based.public.adm_region_cities_nopockets_clipped",
"based.public.adm_region_city_limits_nopockets_ol",
"based.public.adm_region_city_limits_ol",
"based.public.adm_census_tract_clipped",
"based.public.adm_census_tract_unclipped",
"based.public.adm_traffic_analysis_zones_1990",
"based.public.adm_traffic_analysis_zones_2000",
"based.public.adm_district_water",
"based.public.hyd_water_linear",
"based.public.hyd_water_major",
"based.public.hyd_pacific_ocean",
"based.public.hyd_springs",
"based.public.hyd_water_polygon",
"based.public.si_child_care_facilities"]
# Set list of all dataset names to ignore completely because they are obsolete
listOfIgnoredDatasetNames = ["sde.sde.County_Inventory_FC",
"sde.sde.Place_Inventory_FC",
"sde.sde.State_Boundary",
"sde.sde.Postal_Code_Inventory"]
# Start working
for filename in glob2.iglob(os.path.join('N:/*/**/*.mxd')):
print "------------------------------"
print filename
mxd = arcpy.mapping.MapDocument(filename)
# Define function that lists the existing SDE connections
def GetSDEConnectionFileNameList(mapDoc, dataFrame):
#
# Define an empty list of SDE layers in the dataframe
# This is an unnecessary step, but it doesn't hurt anything either
SDELayers = []
#
# Get list of layers in mxd
layerList = arcpy.mapping.ListLayers(mapDoc, "", dataFrame)
#
# Loop through all layers in list
for lyr in layerList:
# Group layers and online service layers don't support DATASOURCE or
# DATASETNAME, so we want to exclude them.
if lyr.supports("DATASOURCE"):
pathToOldSDE = lyr.dataSource
pathTooOldSDE = os.path.dirname(pathToOldSDE)
possibleOldSDE = os.path.basename(pathTooOldSDE)
#
# Check each file for the extension .sde
path, ext = os.path.splitext(possibleOldSDE)
if ext.lower() == ".sde":
#
SDELayers.append(lyr)
#
# Returns a list of all SDE layers for the (mxd, df) pair passed to the
# function
return SDELayers
# Let's do this!
try:
# Look at one data frame at a time
for df in arcpy.mapping.ListDataFrames(mxd):
#
# Get list of SDE connections and layers that use those connections
# in the current data frame
userSDELayers = GetSDEConnectionFileNameList(mxd, df)
#
# Look at one layer in the data frame at a time
for lyr in userSDELayers:
#
# Group layers and online service layers don't support
# DATASOURCE or DATASETNAME, so we want to exclude them.
if lyr.supports("DATASOURCE"):
#
print "nnThe layer's datasource is:", lyr.dataSource
#
# Grab current dataset name
oldDatasetName = lyr.datasetName
#
# Insert cases here
if lyr.datasetName in listOfIgnoredDatasetNames:
# Don't do a swap. Layer is obsolete.
print "ntNow skipping an obsolete layer..."
else:
print "ntNow in LIST case..."
# Get index of old name
nameIndex = listOfOldDatasetNames.index(lyr.datasetName)
# Get new name from index of old name
newName = listOfNewDatasetNames[nameIndex]
# Swap the SDE data source
lyr.replaceDataSource(newSDEFile, "SDE_WORKSPACE", newName, False)
print "nNEW datasource is", lyr.dataSource
#
#
else:
print "nntLayer does NOT support datasetNamen"
del lyr
#
# Save the mxd
mxd.save()
# Error message
except Exception as e:
#
# Print error message
print e.message
#
# Add error message to geoprocessing log file
arcpy.AddError(e.message)
finally:
#
# Delete the mxd variable so there's no lock left on the file
del mxd
print "This script is done. nYou may close the python window."