I need to test processing of files by a process that ingests text files into an Oracle Database. For this I do not care what the data looks like once ingested. I care more about testing the ingestion process which is based on a JAVA application. To give the ingestion process a good workout I generate a ton of files, the more the better. To do this I kludged together a powershell script that does the job. The script generates as many text files as I want using any text source I want. Then the script inserts a random date into each file so that they appear as new files to Oracle. I could use powershell to randomly add anything but I choose dates since there is a date field in the Oracle database.
I found the powershell snippet that generates random dates online a while back and not sure who gets credit, but it is not me. A quick search this morning and I saw several hits in Google for the code.
Basically the script creates a new file appends a random date then cats a source text file and appends that to the new file. Even though each file has the same information they are random because of the random date.
#declare some variables
$SourceText=”c:\path\to\mytext.txt” (edit as needed)
$DestLoc=”c:\path\to\where\files\go (edit as needed)
$myTxt= cat $SourceTxt (I used this because I did not find a simple way to cat the source file right in the code when I need the text. The code is “Add-content $DestLoc\$y.txt $myTxt” I tried using backticks like used in bash to put the output of one command into another command like so: “Add-content $DestLoc\$y.txt `cat $SourceText`” but powershell does not like that.
#set how many files are needed
$Xfiles=10000 (edit as needed)
#set up a loop
for ($i=1; $i -lt $Xfiles; $i++)
{
$y = $i; (on each iteration set $y = current value of $i)
new-item $DestLoc\$y.txt -type file -force (create the new empty file using $y as the file name)
#Generate a random date
[[DateTime]$theMin = "1/1/2008" (start date edit as needed)
[DateTime]$theMax = [DateTime]::Now (end date, for this it is current, edit as needed)
$theRandomGen = new-object random
$theRandomTicks = [Convert]::ToInt64( ($theMax.ticks * 1.0 - $theMin.Ticks * 1.0 ) * $theRandomGen.NextDouble() + $theMin.Ticks * 1.0 )
$dateNow=(new-object DateTime($theRandomTicks))
#Append random date to new file
Add-content $DestLoc\$y.txt $dateNow
#Append target text to new file
Add-content $DestLoc\$y.txt $myTxt
}
Uncluttered Script_________________________________________________________
#declare some variables
$SourceText=”c:\path\to\mytext.txt”
$DestLoc=”c:\path\to\where\files\go
$myTxt= cat $SourceTxt
#set how many files are needed
$Xfiles=10000
#set up a loop
for ($i=1; $i -lt $Xfiles; $i++)
{
$y = $i;
new-item $DestLoc\$y.txt -type file -force
#Generate a random date
[[DateTime]$theMin = "1/1/2008"
[DateTime]$theMax = [DateTime]::Now
$theRandomGen = new-object random
$theRandomTicks = [Convert]::ToInt64( ($theMax.ticks * 1.0 - $theMin.Ticks * 1.0 ) * $theRandomGen.NextDouble() + $theMin.Ticks * 1.0 )
$dateNow=(new-object DateTime($theRandomTicks))
#Append random date to new file
Add-content $DestLoc\$y.txt $dateNow
#Append target text to new file
Add-content $DestLoc\$y.txt $myTxt
}
