Sunday, September 18, 2011

Using rsync to copy/sync two file systems

I've used the rsync command a number of times to synchronize a couple files systems. It's particularly useful for doing backups of one set of files to another. These are some EXTREMELY loose notes about how I did things a couple times that I found in old emails.

Example 1: Replicate on directory to another place on same machine.
cd /data
rsync -a dir1 /dest

Copies all the files from /data/dir1 and drops them in /dest/dir1.

Example 2: Doing on windows with cygwin between two separate windows boxes. Very loose instructions, but a long time ago I was able to make this work between two windows boxes by using a client/server model (e.g. remotely connecting between the two boxes.)

  • install cygwin
  • Map \\axis\backup as X:
  • mkdir -p /cygdirve/x/LaptopBackup/Users-rmills
  • cd C:/Users
  • rsync -a rmills /cygdrive/x/LaptopBackup/Users-rmills
Effectively copies everything from my /Users/rmills directory to the backup directory.

Example 3: Starting from scratch and copying a lot of stuff

On a windows server box, setup K: in /rmills/rsync
  • Create rsync.conf
  • Create rsync.scrt
  • On Server:
    • rsync --verbose --daemon --config=./rsync.conf --no-detach
    • (Take away --no-detach after prove it works.)
  • On Client:
    • rsync -progress -av SomeDirectory rmills@server::vol1
    • (consider "--update" to see if it just copies a few files)
 This basically allows two rsync processes to reach out to each other and copy files.

Friday, September 16, 2011

Parameterizing deployment of ASP applications with VS and MSDeploy

I've worked with a few ASP applications recently where I've wanted to be able to custom-configure the Web Config upon deployment depending on whether it's going to a developer's machine, test server, or production environment. In particular, I'm working on an application now that needs to pull data via SOAP or WCF from a neighboring SharePoint server. The URL of this server will, of course, be different depending on the environment.

There are a variety of situations I wanted to handle.
  1. Local deployment and testing on a development rig
  2. Fully scripted deployment using Web Deploy through a continuous integration server onto our development server
  3. On-demand deployment using the IIS "Deploy > Import Application" wizard
I more-or-less ignored case #1 because this is what developers do every day and they just hack whatever the way to make it work. (Sigh.) I did, however, want to make sure that the parameterization I did continued to work in their environment(s).

For case #2, I've been using msdeploy.exe via the App.deploy.cmd script that appears when you do "Build Deployment Package" from Visual Studio. This can be invoked from a script by simply doing MSBuild /t:Package during build. I'll leave the dynamics of that out of scope for this post. I'm more interested in how you set up the "SetParameters.xml" file properly so that you can feed MSDeploy some scripted parameters during installation to get the Web.Config the way you want it.

For case #3, I've been using the Deploy > Import Applications wizard to pull in the ZIP file produced by the "Package" step of #2. This works well and has a "Enter Application Package Information" page/panel for setting application parameters such as "Application Path" and "Connection String." I wanted to find a way to add parameters to this panel requesting other values.

As it turns out, there are a two primary techniques for building these kinds of configurations. The first is to use "Configurations" in Visual Studio to create additional Web.Config transforms that will substitute values that can be compiled into the applications for different environments. This works well if you know (ahead of time) what the parameters will be in all your configuration environments.

Build Configuration for Web.config

To make this work, you can first create a visual studio "Configuration" in Build > Configuration Manager. Then, create a "New..." configuration.




 I created one called "Staging"


Then, in your Solution Explorer, you use "Add Config Transformations" on the Web.config file.



 This will look at your configurations and create a Web.config transformation file for any new Configurations you have.



 Open your new Web.staging.config file and add an entry:

 
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <appSettings>
     <add key="MySetting" value="My Value"
         xdt:Transform="Replace" xdt:Locator="Match(key)"/>
  </appSettings>
</configuration>

Now, when you select the "Staging" configuration and do a "Build Deployment Package" from Visual Studio, it will automatically replace the setting in your master Web.config with this value. You may have to use Build > Configuration Manager to select the build configuration you want to use (e.g., Staging). You can deploy this package an it will automatically be set properly. This is especially useful for Database connection strings that tend to be fairly well known in most development and testing environments, as long as you're not worried about checking those values into your version control system. I used the following for a DB connection string for the ASP application we were building:

 
  <connectionStrings>
    <add name="ApplicationServices"
      connectionString="data source=TestDBServer;Integrated Security=SSPI;AttachDBFilename=|DataDirectory|\aspnetdb.mdf;User Instance=true"
      xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
  </connectionStrings>

Parameters.xml

The second way to parameterize your web config is to establish a "parameters.xml" file for your application that creates these transforms in an externalized way that can be set at deployment time. This is the technique that actually works for cases #2 and #3 above. By setting up parameters.xml, I get a entry to appear in the SetParameters.xml file as well as in the deployment wizard within IIS manager. I'll focus on that for the rest of this post.

First, create a new XML file called "parameters.xml" and store it in the root of your project. Put the following entries to modify your parameters:


 
 <parameters>
  <parameter name="External SP Server Url"
             description="URL of external SharePoint server. Example: http://externalsp.portal.local "
             defaultValue="http://localhost"
             tags="">
    <parameterEntry
        kind="XmlFile"
        scope="obj\\Debug\\Package\\PackageTmp\\Web\.config$"
        match="//appSettings/add[@key='Portal.ExternalSPServerUrl']/@value" />
  </parameter>
</parameters>

This will generate the proper "External SP Server Url" entry in your SetParameters.xml file for scripting as well as place it on the wizard when you deploy with IIS. This replaces a key="Portal.ExternalSPServerUrl" in configuration/appSettings section of the web.config. It took me a while to get the "match" criteria set properly to find it and set the value.

Both these solutions work together -- parameters.xml and Web.config configuration files. One thing to note is that the "defaultValue" in the parameters.xml file will trump whatever settings you have in the web.config templates, so you need to provide a proper value during installation.

References

I found the following links valuable in figuring this out:
  • http://msdn.microsoft.com/en-us/library/ff398068.aspx - how to use parameters to configure deployment
  • http://vishaljoshi.blogspot.com/2009/03/web-deployment-webconfig-transformation_23.html - web config transformation using configurations
  • http://msdn.microsoft.com/en-us/gg454290 - transforming web config for deployment

Thursday, September 15, 2011

Deploying ASP web application with command line msdeploy without deleting all files on server

.
This mimicks the IIS wizard prompt "No, just append the files in the application package to the destination" on the "Overwrite Existing Files" panel. I used the "-enableRule:DoNotDeleteRule" for msdeploy.exe to accomplish this.


First, Use "Package" option within Visual Studio 2010 or via MSBuild.exe on the command line. This produces a "MyApp.deploy.cmd" along with a matching "MyApp.SetParameters.xml" that goes with your MyApp.ZIP file.

Then, simply invoke the CMD file to deploy the application using the "enableRule" parameter.

C:\>call MyApp.deploy.cmd /Y /m:myhost /u:my_user /p:my_password -enableRule:DoNotDeleteRule -debug -setParamFile:C:\MyPath\MyApp.SetParameters.xml

This will prevent configuration files such as web.config from being deleted as it deploys the application over top of the previous installation.

This has a few assumptions:
  1. You've already gone through the whole set of configuration plumbing needed to set up the MSDeploy command to properly work for remotely deploy the application to IIS server. 
  2. You already have things working such that you can deploy the ZIP file using IIS manager with Deploy > Import Application and stepping through the wizard.