I was building some HDInsight Azure Data Factory pipelines the other day and the Hive scripts were not doing what I wanted to them to do (although they were doing what was being asked).I needed to isolate the Hive scripts to see what they were doing at each stage of the script. I didn’t want to have to invoke the whole ADF just to look at the Hive section. I was in Visual Studio already so troubleshooting was easy from here. Here’s how I solved it.
Another quirk of my Hive Script was that I was passing in values from the activity to the Hive Script itself. I knew what the values were that I was using but didn’t want to change the script to use hard coded values instead of the reference to the config. Here’s what I mean
I wanted to pass in my values as parameters to the script in Visual Studio. No problem. Once you have finished writing your query
Hit Submit | Advanced
Now you can enter your configuration key value pairs
This for me was a really quick and useful way of troubleshooting my Hive scripts. Hope you find it useful.