Localize Objects with PowerShell and VSCode

In this article I will explain how you can use PowerShell to extract the right objects and merge those objects, and how to use Visual Studio Code to resolve most of the conflicts in the merged objects.

In a previous article, I explained how you can use PowerShell to create the environments. The script in this article actually use the variables that are assigned in the other one. What I should really do is create a configuration file and load that from both scripts. I wanted to share what I have so far though, without a handy configuration file but hopefully helpful nonetheless.

We start off with the 4 environments that were created in our last article: ORIGINAL, MODIFIED, TARGET, and RESULT. The first two environments contain the standard W1 and the ISV product. Target and RESULT are identical at this point, and they both contain the standard NA localization. All environments are in the same build number (NAV 2017 CU2 in this case). Because I have a developer license that has insert rights in the ISV number range, my strategy is to merge the product modifications into the NA database. If you do not have those insert rights, then a better strategy would be to merge the NA localization into the product environment instead. Microsoft has recently added insert capabilities for the standard object ranges to regular developer licenses, for this particular purpose. As I was working through the conflict objects of this assignment, I was thinking that this may even be the best default strategy anyway.

Due to a limited amount of time, and a limited amount of PowerShell skill, I decided to approach this task pragmatically:

  • Manually move the ISV specific objects to the RESULT environment
  • Use PowerShell to export all the objects
  • Manually eliminate the unmodified objects
    • NOTE: as I learned later, Waldo’s PowerShell modules actually have logic to remove these after the merge, so this can be scripted as well
  • Use PowerShell to merge the objects
  • Use Visual Studio Code (AKA VSCode) to resolve the conflicts as much as possible
  • Use PowerShell to join the objects
  • Use C/SIDE to resolve any remaining conflicts

The conflict resolution is something that has to be done manually. Everything else can be scripted in PowerShell. During this process I’ve asked Waldo for some help, and he explained that most of what I am doing here is already part of the merge sample scripts. In the Cloud Ready Software PowerShell module, there is a folder called “PSScripts”, and in that folder you will find a large number of scripts that you can use as an example to get started on your task. As you gain experience in using PowerShell, you will recognize a lot of useful features in those scripts, and you can modify them to your specific needs.

Alright, on to the details. The first part is to use PowerShell to extract the objects into their own folders. I also added commands to create the folder structure itself:

At this point, you will have an “Objects” folder in your working folder, with full object files for ORIGINAL, MODIFIED, and TARGET. Those object files have then been split into individual object files in the ORIGINAL, MODIFIED, and TARGET folders. Before using PowerShell to merge those objects, I used a text compare tool (I like Scooter Software’s Beyond Compare) to eliminate unmodified objects. Remember, I only want to work on the modified objects, so I don’t have to worry about getting confused by objects that were not changed for the ISV product.

Now that we only have modified objects in all three object folders, we’ll use the merge Cmdlet to do the actual merging of the objects.

If you notice, the output of the “Merge-NAVApplicationObject” Cmdlet is loaded into the variable “$MergeResult”. This object is then piped into the “Merge-NAVApplicationObjectProperty” Cmdlet. The first Cmdlet is a standard NAV PowerShell Cmdlet, and the second one comes with the Cloud Ready Software modules. The standard merge Cmdlet does not merge the Version List property, it simply takes the Version List from one of the three environments. We can have a discussion about whether the Version List is even important anymore, especially if you use Source Code Management. The reality is that most NAV developers depend on proper tags in the Version List, so it is useful to merge those as well. All you need to do is specify the prefixes that you want to merge (In my case NAVW1 and NAVNA) and it will find the highest value for those prefixes. All other prefixes will be simply copied into the RESULT objects.

At this point, we have all merged files in the RESULT folder. This includes the objects that were merged successfully, but also the objects that the merge Cmdlet could not resolve, for instance when code was added at the same position in MODIFIED and in TARGET. Since we are going to have to learn how to use VSCode, I decided to use that to resolve the conflicts.

With VSCode installed, you can open the RESULT folder and see the content of this folder inside the file browser at the left hand side. In the upper right hand corner is a button that you can use to split the screen into two (and even three) editor windows. This is very useful for resolving merge conflicts, because you can open the conflict file in one side, and the object file in the other side. You are editing the actual object file here, so you may want to take a backup copy of the folder before you get started.

Now what you must understand is that the conflict files only contains the pieces that the merge Cmdlet could not figure out. In a total of 860 objects, with 6956 individual changes, it was able to merge 96.4% of those changes. An object that may have 4 conflicts can also have a ton of other changes that the merge Cmdlet merged successfully. All YOU have to do is focus on the ones that need manual attention. For instance, codeunits 80 and 90 had a TON of modifications, but it only needed help with 3 of them.

I had a total of 115 conflict files, and I could completely resolve 112 of them in VSCode. I made a note of the few that remained, and decided to import those unchanged into C/SIDE, so I can finish those off in the proper IDE.

The last PowerShell command that I used is to create a single object file, which can then be imported into C/SIDE. Then I finished the last few remaining object files, and was able to compile all objects from there.

I’ve done many of these merges completely manually. To say that I was skeptical that PowerShell would do a good job is putting it mildly. I flat out did not trust the merge Cmdlet, surely they could not do as good a job as I could do. I was wrong. I checked a bunch of objects to see if I could find any mistakes and I could not find any. Not only did the merge Cmdlet do a fine job at merging the objects, it did so in about 5 minutes flat.

You can download the scripts here.

Instead of having to manually merge almost 900 objects, all I had to do was focus on the conflicts. Usually, a vertical merge like this would take me anywhere from 2 to 4 weeks, and I was able to finish this one in less than two days. Figuring out the PowerShell scripts took me much longer, but I will be able to use those for the next merge task.

Create NAV Environment with PowerShell

If you kind of know about PowerShell, and you want to use it more, but you don’t really know where to start, then you should read on. I am going to explain to you how you can use PowerShell to create the environments that you need to localize a product.

One of the things that always seem to fall on my plate at my job is to take an ISV product and merge that into the North American localization. I call it ‘localizing a product’. It is not, because localizing a product is much more than simply merging the objects, but it’s what you have to do first. I’ve done a bunch of these manually, and I kind of enjoy the almost mindless nature of the task of working my way through hundreds of objects (sometimes even thousands, I once ‘localized’ a product that had more objects than standard NAV itself).

This time around, I wanted to use PowerShell to automate as much as I could. Lucky for me, I attended Waldo’s “PowerShell Black Belt” workshop at NAV Techdays (read my review of this fantastic event here) and I should have all the tools to get this started.

The first thing you need is to install Waldo’s PowerShell modules, I will be using those for just about everything. I could figure out how to script all of this stuff myself, but why would I if Waldo already did that for us, and he is sharing his scripts. For instructions read Waldo’s blog here.

This post focuses on creating the environments themselves, and for this task you really need just a few things:

  • A SQL backup for the product in the W1 version of NAV, and make sure that you know exactly which build this was developed in. For instance, the product that I am working with was provided as a SQL Backup, and it was developed in NAV 2017 CU2.
  • The DVD’s for standard W1 and NA for the same NAV build.
  • A Development license that has insert rights for the product’s number range. If you only have a regular developer license, or if the ISV refuses to provide you with a license (this happens more often than not) then you will have to merge NA into the product database. This works exactly the same, you just have to figure out which environments are your ORIGINAL, MODIFIED and TARGET.

Use the NA DVD that you just downloaded and install the NA Version with a demo database. I like to give the database a meaningful name, so I called it ‘NAV2017NACU2’. This is all the manual installing that we will do, everything else is PowerShell baby! Actually, you could even use Waldo’s PowerShell modules to do the install as well. You could even write a script that downloads the DVD for you as well, all using Waldo’s PowerShell modules.

If you understand the mechanics of localizing a solution, you will know that we need 4 databases in total: standard W1, Product database in W1, standard NA, and a copy of the standard NA database that will become the product database. In this case, standard W1 is ORIGINAL; product W1 is MODIFIED; standard NA is TARGET and product NA is RESULT. This will become important when we do the merge, which will be another blog post.

Now the goal is to have a re-usable script, so we’ll use variables instead of having to re-type values multiple times, as you can see in the picture.

Remember, I installed standard NA. I created the working folder and placed the three backup files into the ‘Backups’ folder. I like to keep things together, but I can imagine having a network location for the standard backup files. The nice thing about using these variables is you can set them however you need them :). My database server is called KERPLUNK, which is just a regular unnamed instance on my VM called KERPLUNK. You could also use a demo installation and then you’d have to set it to ‘KERPLUNK\NAVDEMO’. I will use the 4 names for the database names as well as the service tier names.

The rest of the script is surprisingly simple to put together, here it is:

The reason why I start with the ‘Copy-NAVEnvironment’ step is because this script turns on port sharing for both server instances, so I don’t have to take care of that step myself. This is one of Waldo’s script that looks at the first server instance and creates a new one just like it. It creates a SQL Backup from the service instance, restores that into a new database, and finally it creates a new server instance for that new database, and it enables port sharing by default. The ‘New-NAVEnvironment’ is also one of Waldo’s scripts, and it does all the heavy lifting of restoring the database, creating the service tier and setting up port sharing. The full script took maybe 2-3 minutes to run for me, not bad for something that used to take all morning.

This same script could also be used to create your environments for developing extensions. For that you only need a standard database for ORIGINAL and a development database for MODIFIED, both of which will then be used to create the DELTA files for the extension package. If you look at the screenshot of my environment you’ll see an APPDEV and an APPTEST database. Both of these were originally created as copies of the standard database, also using PowerShell.

I’m planning to also put this into a video but I actually have this work to deliver so I’ll focus on that first. Up next is getting the objects out and comparing them to create a set of merged objects. Stay tuned!

Update 4/22/2017: added link to the new article about localizing the objects, and you can download the scripts here.

NAV Techdays 2016 Recap

This past week was the NAV Techdays event in Antwerpen. If you don’t know what this is, head out to their website and read the ‘About’ section. This year’s edition was I think the 7th time, and it was the first time that there were more than 1,000 attendees! In my opinion, this event is BY FAR the best technical event for NAV, head and shoulders above any other event, not in the least because there are two days of pre-conference workshops.

Last year was the first time I had gone to NAV Techdays, and after the truly exhausting three days, I was utterly satisfied and I promised myself that I would do everything in my power to come back every year going forward.

My highlights:

  • Pre-conference workshops. I had signed up for Black Belt Powershell by Eric ‘Waldo’ Wauters, and for Deep Dive into Events/Extensions by Arend-Jan Kauffmann. Both days were very intense, there was a TON of new information for me, and I was utterly done at the end of both days. The knowledge that I have gotten out of these two workshops will help me stay relevant as a technical person in the NAV world. These two workshops alone were worth traveling to Europe for.
  • Extensions related sessions – I made it a point to attend as many sessions about extensions as possible, since I believe it is the most important technological development in our industry this year. Especially Gunnar’s session about migrating to Events and Waldo’s session about Extensions Best Practices were eye openers
  • PowerBI and NAV by Steven Renders was absolutely INCREDIBLE. The sheer amount of information that this guy put into his session was just unbelievable. I bet he could fill two days of proper training about all the things that he showed us. Well done Steven!

I would have liked to see the sessions about Office 365 and the design patterns session, but I can’t be in two places at the same time. Lucky for me (and everyone else) Luc records all of the sessions and puts them on YouTube. He even puts the videos in individual playlists per year, for instance you can see all videos for 2016 here.

NAV Techdays is my favorite event. The sessions, the pre-conference workshops, the super-comfortable venue, the amenities, the production, the expo area, meeting my kind of peeps in person. It feels like this is “my” event, and I hope to be able to go each and every year from here on out.

Update 11/20/2016: Slide decks available here

Update 11/23/2016: Added links to YouTube videos