To use, cut and paste into a new html doc to test it, if you name your html doc anything other than DynamicHiddenField.html, then update the form's action as need be.
The hidden Field appears when you click on the 'Appear' button. Try submitting with and without hitting the Appear button. Since it's method is a 'get' you can see the value of the field in the url after a submission (or lack of if you didn't hit 'Appear'). This ablitity to test it in a submission will show you the appearance of the field is fully functional for submitting data with the form.
document.getElementById('testAreaHidden').innerHTML ="<input type='hidden' name='hiddenField1' value='New Hidden Field'>";
alert("new button function");
<form method="get" action="DynamicHiddenField.html">
<input type="button" value="Appear" onClick="fieldAppear()">