Tuesday, October 25, 2011

Repairing 'svnsync: malformed file' error with subversion


I recently had a sync or other failure of svnsync which left me with a bum repository. Every time we tried to do svnsync, it reported "svnsync: malformed file." I got the same error with all svnadmin access.

I found this link: http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1892873&orderBy=createDate&orderType=desc and provided some good clues. I noticed that my file in db\revprops\0 had junk in it instead of normal stuff. It should have looked like this:

K 8
svn:date
V 27
2010-07-21T02:55:12.203323Z
K 17
svn:sync-from-url
V 45
https://portal.secureci.com/svn/secureci
K 18
svn:sync-from-uuid
V 36
b443ad56-1e36-4a4f-97ef-7144cc153033
K 24
svn:sync-last-merged-rev
V 1
0
END


What I did to fix it:

  • cd c:/temp
  • svnadmin create svn-sync2
  • Copy hooks/pre-revprop-change.bat file from broken repository to new one
  • svnsync --username svnsync initialize file:///c:/temp/svn-sync2 https://portal.secureci.com/svn/securci
  • Copy the db/revprops/0/0 file from the new repos to the broken one
  • Edit the 0 file and fix the 'sync-last-merged-rev' entry to have the correct number.
    • Look in db/revprops/2 (or highest number) and locate the most recent numbered file (e.g., 2437)
    • Change '0' value below "V 1" to be "2437"
    • Change the V1 line to be "V 4" for 4 character length
  • Run 'svnadmin verify .' in the main directory of your broken repository to make sure that it has everything intact. (This may take a little while.)
  • Run 'svnsync --username svnsync synchronize file:///c:/SecureCIdata/svn/repos/secureci' and make sure synchronization goes smoothly
That's it. Syncing should work normally now.

Wednesday, October 5, 2011

DOS batch file stderr redirection

I never remember the whether the syntax of DOS file redirection is similar to normal Unix file redirection. It is. So I'm writing it down where I can find it.

CALL .\somescript.bat >> logfile.log 2>&1

Of note, I recently learned that redirecting output from a PowerShell script does NOT work very well and gives file handle errors. That's another story, entirely.

Common DOS batch file error handling mistakes I make

I write a decent amount of batch scripts now days to automate things on windows server boxes. Unfortunately, I don't do it quite frequently enough to remember all the crazy syntax of DOS commands. One of these errors is error handling. These are the things I commonly need to deal with.

I found this page (and site) very helpful for all DOS related stuff: http://www.robvanderwoude.com/errorlevel.php

  1. Use "IF %ERRORLEVEL% NEQ 0 SET MYERROR=1" to record whether the previous command resulted in an error code. Alternately, you can check for "IF ERRORLEVEL 1 ..." if you want to look for a specific error level.
  2. Be cautious of manipulations with ERRORLEVEL. It's not really an environment variable like other variables. In particular, do NOT ever use "set ERRORLEVEL=5" or similar. It will corrupt any further use of %ERRORLEVEL% syntax by fixing it at a value.
  3. Use "CALL FOO.BAT" instead of just "FOO.BAT" when calling subscripts. Otherwise, when that script completes it will not return to the current script.
  4. Use "EXIT /B 1" to return an exit code from your script. If you use "EXIT 1" it will exit the entire command shell, including closing your current window if it's running in one.
  5. Consider using SETLOCAL and ENDLOCAL within your script to prevent temporary environment variables from carrying through to outer shells.

Below is an example script.


:: Sample script with some error handling
SETLOCAL
SET MYPARAM=%1
if "%MYPARAM%"=="" goto :USAGE

CALL .\childscript.bat %MYPARAM%
if %ERRORLEVEL% NEQ 0 set MYERROR=1

echo.
echo Finished child script. Handling errors now...

if %MYERROR%==1 GOTO :ERROR

echo It worked!
ENDLOCAL
set SOME_EXTERNAL_VARIABLE=1


GOTO :EOF
:USAGE
echo Please provide a command line parameter.
EXIT /B 2


:ERROR
echo It didn't work, dude.
EXIT /B 1 

Update: I was wrong about the ERRORLEVEL syntax earlier, so I updated after some testing.

Monday, October 3, 2011

Change Windows 7 default logon screen background

When you change your desktop background in Windows 7, it does not change the background for the logon screen. Since I have multiple machines connected to a KVM on my setup, I like to have different colors to signify which machine I'm actually looking at.

Windows 7 doesn't provide a good mechanism through the UI for changing this. To do it, you need to change a registry key and store your background as the correct file.

I found this post which was helpful: http://www.kodyaz.com/articles/change-windows-7-logon-screen-background-using-registry-oobe.aspx

Bascially,
  1. Navigate to this registry key path with regedit.exe: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\
    Microsoft\Windows\CurrentVersion\Authentication\LogonUI\Background
  2. Make sure the OEMBackground DWORD key is set to 1.
  3. Navigate to C:\Windows\System32\oobe\info\backgrounds directory
  4. There will probably already be a bunch of backgrounds there. Pick the one you want and save it as 'backgrounddefault.jpg'
That's it. You're done. Hit Windows-L to lock your screen and you should see the new background file.

Sunday, September 18, 2011

Using rsync to copy/sync two file systems

I've used the rsync command a number of times to synchronize a couple files systems. It's particularly useful for doing backups of one set of files to another. These are some EXTREMELY loose notes about how I did things a couple times that I found in old emails.

Example 1: Replicate on directory to another place on same machine.
cd /data
rsync -a dir1 /dest

Copies all the files from /data/dir1 and drops them in /dest/dir1.

Example 2: Doing on windows with cygwin between two separate windows boxes. Very loose instructions, but a long time ago I was able to make this work between two windows boxes by using a client/server model (e.g. remotely connecting between the two boxes.)

  • install cygwin
  • Map \\axis\backup as X:
  • mkdir -p /cygdirve/x/LaptopBackup/Users-rmills
  • cd C:/Users
  • rsync -a rmills /cygdrive/x/LaptopBackup/Users-rmills
Effectively copies everything from my /Users/rmills directory to the backup directory.

Example 3: Starting from scratch and copying a lot of stuff

On a windows server box, setup K: in /rmills/rsync
  • Create rsync.conf
  • Create rsync.scrt
  • On Server:
    • rsync --verbose --daemon --config=./rsync.conf --no-detach
    • (Take away --no-detach after prove it works.)
  • On Client:
    • rsync -progress -av SomeDirectory rmills@server::vol1
    • (consider "--update" to see if it just copies a few files)
 This basically allows two rsync processes to reach out to each other and copy files.

Friday, September 16, 2011

Parameterizing deployment of ASP applications with VS and MSDeploy

I've worked with a few ASP applications recently where I've wanted to be able to custom-configure the Web Config upon deployment depending on whether it's going to a developer's machine, test server, or production environment. In particular, I'm working on an application now that needs to pull data via SOAP or WCF from a neighboring SharePoint server. The URL of this server will, of course, be different depending on the environment.

There are a variety of situations I wanted to handle.
  1. Local deployment and testing on a development rig
  2. Fully scripted deployment using Web Deploy through a continuous integration server onto our development server
  3. On-demand deployment using the IIS "Deploy > Import Application" wizard
I more-or-less ignored case #1 because this is what developers do every day and they just hack whatever the way to make it work. (Sigh.) I did, however, want to make sure that the parameterization I did continued to work in their environment(s).

For case #2, I've been using msdeploy.exe via the App.deploy.cmd script that appears when you do "Build Deployment Package" from Visual Studio. This can be invoked from a script by simply doing MSBuild /t:Package during build. I'll leave the dynamics of that out of scope for this post. I'm more interested in how you set up the "SetParameters.xml" file properly so that you can feed MSDeploy some scripted parameters during installation to get the Web.Config the way you want it.

For case #3, I've been using the Deploy > Import Applications wizard to pull in the ZIP file produced by the "Package" step of #2. This works well and has a "Enter Application Package Information" page/panel for setting application parameters such as "Application Path" and "Connection String." I wanted to find a way to add parameters to this panel requesting other values.

As it turns out, there are a two primary techniques for building these kinds of configurations. The first is to use "Configurations" in Visual Studio to create additional Web.Config transforms that will substitute values that can be compiled into the applications for different environments. This works well if you know (ahead of time) what the parameters will be in all your configuration environments.

Build Configuration for Web.config

To make this work, you can first create a visual studio "Configuration" in Build > Configuration Manager. Then, create a "New..." configuration.




 I created one called "Staging"


Then, in your Solution Explorer, you use "Add Config Transformations" on the Web.config file.



 This will look at your configurations and create a Web.config transformation file for any new Configurations you have.



 Open your new Web.staging.config file and add an entry:

 
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <appSettings>
     <add key="MySetting" value="My Value"
         xdt:Transform="Replace" xdt:Locator="Match(key)"/>
  </appSettings>
</configuration>

Now, when you select the "Staging" configuration and do a "Build Deployment Package" from Visual Studio, it will automatically replace the setting in your master Web.config with this value. You may have to use Build > Configuration Manager to select the build configuration you want to use (e.g., Staging). You can deploy this package an it will automatically be set properly. This is especially useful for Database connection strings that tend to be fairly well known in most development and testing environments, as long as you're not worried about checking those values into your version control system. I used the following for a DB connection string for the ASP application we were building:

 
  <connectionStrings>
    <add name="ApplicationServices"
      connectionString="data source=TestDBServer;Integrated Security=SSPI;AttachDBFilename=|DataDirectory|\aspnetdb.mdf;User Instance=true"
      xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
  </connectionStrings>

Parameters.xml

The second way to parameterize your web config is to establish a "parameters.xml" file for your application that creates these transforms in an externalized way that can be set at deployment time. This is the technique that actually works for cases #2 and #3 above. By setting up parameters.xml, I get a entry to appear in the SetParameters.xml file as well as in the deployment wizard within IIS manager. I'll focus on that for the rest of this post.

First, create a new XML file called "parameters.xml" and store it in the root of your project. Put the following entries to modify your parameters:


 
 <parameters>
  <parameter name="External SP Server Url"
             description="URL of external SharePoint server. Example: http://externalsp.portal.local "
             defaultValue="http://localhost"
             tags="">
    <parameterEntry
        kind="XmlFile"
        scope="obj\\Debug\\Package\\PackageTmp\\Web\.config$"
        match="//appSettings/add[@key='Portal.ExternalSPServerUrl']/@value" />
  </parameter>
</parameters>

This will generate the proper "External SP Server Url" entry in your SetParameters.xml file for scripting as well as place it on the wizard when you deploy with IIS. This replaces a key="Portal.ExternalSPServerUrl" in configuration/appSettings section of the web.config. It took me a while to get the "match" criteria set properly to find it and set the value.

Both these solutions work together -- parameters.xml and Web.config configuration files. One thing to note is that the "defaultValue" in the parameters.xml file will trump whatever settings you have in the web.config templates, so you need to provide a proper value during installation.

References

I found the following links valuable in figuring this out:
  • http://msdn.microsoft.com/en-us/library/ff398068.aspx - how to use parameters to configure deployment
  • http://vishaljoshi.blogspot.com/2009/03/web-deployment-webconfig-transformation_23.html - web config transformation using configurations
  • http://msdn.microsoft.com/en-us/gg454290 - transforming web config for deployment

Thursday, September 15, 2011

Deploying ASP web application with command line msdeploy without deleting all files on server

.
This mimicks the IIS wizard prompt "No, just append the files in the application package to the destination" on the "Overwrite Existing Files" panel. I used the "-enableRule:DoNotDeleteRule" for msdeploy.exe to accomplish this.


First, Use "Package" option within Visual Studio 2010 or via MSBuild.exe on the command line. This produces a "MyApp.deploy.cmd" along with a matching "MyApp.SetParameters.xml" that goes with your MyApp.ZIP file.

Then, simply invoke the CMD file to deploy the application using the "enableRule" parameter.

C:\>call MyApp.deploy.cmd /Y /m:myhost /u:my_user /p:my_password -enableRule:DoNotDeleteRule -debug -setParamFile:C:\MyPath\MyApp.SetParameters.xml

This will prevent configuration files such as web.config from being deleted as it deploys the application over top of the previous installation.

This has a few assumptions:
  1. You've already gone through the whole set of configuration plumbing needed to set up the MSDeploy command to properly work for remotely deploy the application to IIS server. 
  2. You already have things working such that you can deploy the ZIP file using IIS manager with Deploy > Import Application and stepping through the wizard.