Tuesday, October 8, 2019

How to create SSL certificate including a self-signed root certificate using Powershell

This blog post gives an overview of the steps needed to create self-signed certificate that can be used for SSL communication including a self-signed root certificate.

Context:
I wanted to have a HTTPS server that uses a wildcard certificate that was signed by my own (self-signed) root certificate. The HTTPS server is a small executable that is home made and running under the current user. The goal was to use the Chrome browser to fetch a page from the HTTP over TLS without the browser complaining about security issues.

I thought this would be a straight forward exercise. However it cost me some effort to get the right information and to get things working. I started out using the "makecert" tool. However I was unable to add a Certificate Alternative Name entry to the certificate. This resulted in an error from Chrome.

The overcome this I switched to PowerShell and finally got things to work. The credit goes to the blog entry posted on the InfiniteLoop.IO site: Powershell – Self-signed Certificate via Self-signed Root CA


Here are the steps:

Step 1 – Create the root certificate

The following code creates the certificate and installs it in the local machine certificate store. The code needs to run from with administrator rights.

$params = @{
  DnsName = "myroot"
  KeyLength = 2048
  KeyAlgorithm = 'RSA'
  HashAlgorithm = 'SHA256'
  KeyExportPolicy = 'Exportable'
  NotAfter = (Get-Date).AddYears(5)
  CertStoreLocation = 'Cert:\LocalMachine\My'
  KeyUsage = 'CertSign','CRLSign' #fixes invalid cert error
}
$rootCA = New-SelfSignedCertificate @params

Step 2 – Create the server cert signed by the new root

In this step we are going the create the server certificate that is signed by our root certificate. We first need to fetch the root certificate from the certificate store.

$rootCA = ( Get-ChildItem -Path cert:\LocalMachine\My | Where-Object { $_.Subject -Match "myroot" } )

After having a handle on the root certificate we can create the server certificate.

$params = @{
  DnsName = "*.mysite.test.com"
  Signer = $rootCA
  KeyLength = 2048
  KeyAlgorithm = 'RSA'
  HashAlgorithm = 'SHA256'
  KeyExportPolicy = 'Exportable'
  NotAfter = (Get-date).AddYears(2)
  CertStoreLocation = 'Cert:\LocalMachine\My'
}

$vpnCert = New-SelfSignedCertificate @params

Step 3 – Add self-signed root to trusted root certificate store of current windows client

The machine on which the client (browser) is running needs to validate the certificate provided by the server. For this it needs our root certificate.

Again we first need to fetch the root certificate

$rootCA = ( Get-ChildItem -Path cert:\LocalMachine\My | Where-Object { $_.Subject -Match "myroot" } )

Next we export the certificate to a file so we can install it on the client machine(s)

Export-Certificate -Cert $rootCA -FilePath ".\myroot.cer"

Import on the client machine(s). The file contains only the public key
Import-Certificate -CertStoreLocation 'Cert:\LocalMachine\Root' -FilePath ".\myroot.cer"


Step 4 – Export server certificate as .pfx

The server certificate can be exported to file so we can store it in a safe place or to import if we want to run the server on a different machine. Please fill in your own password.

Export-PfxCertificate -Cert $vpnCert -FilePath '.\server.pfx' -Password (ConvertTo-SecureString -AsPlainText 'password' -Force)

The pfx contains the public and private key so please store in a safe place.


Step 5 – Check Result and update access rights

You can check the results by starting the certlm.msc (Certificates Local Machine).

For my setup we need to give access rights to the certificate private key for the account under which the HTTP server is running. This can be done by right clicking the certificate (in certlm.msc) => all tasks => manage private keys.






Sunday, March 11, 2012

Reading up on WinDbg and XPerf

I like to share some interesting articles I found this week. The first article is "How Understanding Assembly Language Helps Debug .NET Applications". It was published by Sasha Goldshtein on Codeplex. Also checkout his blog it contains lots of useful information on .NET.

Another nice post by Roberto Alexis Farah discusses WinDbg and how to display the COM reference held by a RCW. So for those of you who use COM from a managed app this can be helpful when debugging problems.

Then there was a post on the NTDebugging Blog about Global Atom Table leaks. This blog entry guides you through a debug session to debug this kind of problems.

Mark B. Friedman a veteran in the field of performance shares some of his insight on Processor utilization and Xperf on his Performance by Design blog. Mark describes in his October post the details how the ETW CSwitch kernel event should be interpreted. Here is a reference to the MSDN library describing the CSwitch event.

Sunday, February 26, 2012

Event Tracing For Windows Resources

One of the great things about Windows 7 (2008/Vista) is Event Tracing for Windows (ETW). This blog entry contains a collection of pointers to websites and blogs that get you started using ETW.

Overview and Concepts
The NTDebugging blog gives a nice introduction to ETW and its tools. You can find the the first of three articles here: ETW Introduction and overview  .

MSDN Magazine also has a number of publication that are worthwhile reading:
- Improve Debugging And Performance Tuning With ETW

Here are some other good resources on ETW:
- Bruce Dawson (Random Ascii) has a nice intro on ETW traces
- Ashley Whetter has a blog with several pointers

Concepts (see also the Programmers Guide to Eventing):
Providers - Providers: applications that contain event tracing. There are 2 types: manifest-based providers and classing providers.
Events - event/action of interest that can be logged.
Channels - Providers can publish events to channels. Channels are targeted at different roles. Admin and Operational channels target IT professionals and Administrators and are enabled by default, while Analytic and Debug Channels are more in depth, and not usually enabled by default.
Consumers -  display/process events generated by a provider
Controller -  a component to control ETW e.g. enable of disable tracing of events
Keywords (Flags) - specify functional sub-components (HTTP, COOKIES, CONNECTION). Keywords can be used to filter on functions.
Levels - level of detail of the event need to be logged (Error, Informational, Verbose, etc).

Examples
- List all the providers installed on the machine which have 'kernel' in their name:
logman query providers | findstr /i "kernel"

- List all of the user-mode ETW providers for a given process (using its PID). Checkout your own application and you can see which (Microsoft) 'Provider' are available even if you did not instrument your application itself.
logman query providers -pid <PID>

- Show details on a given provider. The example shows the events for this provider including the different keywords and levels.
logman query providers "ntfs" 

- How to find what tracing is available, and its format you can use the wevtutil to get full publishing event and message information
wevtutil get-publisher Microsoft-Windows-WinInet /getevents /getmessage

- Show and enable/disable different providers that are running on your box.
Goto Computer Management => EventViewer => Application and Service logs (user the option Show Analytic and Debug Logs under the view menu). Under "Application and Services logs" you find a list of providers and the different channels they provide. You can enable the events for a channel by clicking the channel and enabling it via the right mouse button.

- Dump the information to a CSV file
tracerpt c:\windows\system32\winevet\logs\Microsoft-Windows-WinINet%4Analytic.etl –of CSV –o c:\temp\Microsoft-Windows-WinINet%4Analytic.csv
You can also use Powershell (v2) see get-help get-winenvent

- Get information on a ETW sessions on the box
logman query -ets

- Get detailed information on a ETW session
logman query -ets <session name>

- Start and stop a kernel trace for data collector set named "NT Kernel Logger" for a given set of keyword, buffer size 1024 MB, 20 buffers in circular mode writing to kernel.etl (max file size 20 MB
logman start "NT Kernel Logger" -p "Windows Kernel Trace" (process,thread,disk,isr,dpc,net,img,registry,file) -bs 1024 -nb 20 20 -mode Circular -o kernel.etl -ct perf -max 20 -ets
logman stop "NT Kernel Logger" -ets
- Starting and stopping a kernel session using Xperf (xperf -providers kernel gives an overview of the predefined kernel groups).
xperf -providers kernel
xperf -on DiagEasy
xperf -d kernel.etl 

- Information on how to configure and start an event tracing session programmatic ally can be found here: Configuring and starting and Event Tracing Session
Instrumenting your application
At this point you are hopefully convinced that ETW rocks. You might even consider to add ETW instrumentation to your own application. If so then keep on reading. Here are some pointers information that I found usefull.

To get started take a look at this article: Improve Debugging And Performance Tuning With ETW

First step to instrumenting the code is to write a manifest. How you do this can be found her:
Writing an instrumentation manifest

The MC.EXE tool (part of the SDK) can be used to converts the manifest into a C# wrapper class (can also generate C++ code). You can instrument you code by calling methods on this class. Check out the following guide Programmers Guide to Eventing

You should name the event providers by using the following convention (see Programmers Guide to Eventing):

       <company_name>-<product_name>-<component_name>


The name forms the path in the EventViewer (under Applications and Services Logs). For Debug and Analytic channels you need to enable (view => Show Analytics and Debug Logs). So when using this name convention and defining channels in you manifest an entry in the EventViewer will show up. From there you can enable/disable the log. Cyclic logs however still seem to have some issues (see kb2488055).

After you have instrumented you code you might want to do automatic analysis (e.g. gather statistics). For this there is a library which can be used to parse the ETL file. It comes with the PerfMonitor tool (see my previous post). You can find the library here: TraceEvent. The steps to do some basic etl file parsing are:
- Create an instance of the ETWTraceEventSource class
- Register a callback for a trace event
- Invoke the Process method

You can also dump the etl content to e.g. an XML file using the tracerpt tool. For example:

tracerpt <myetlfile.etl> -import <mymanifest> -y


The tracerpt tool will write a the etl content to (by default) dumpfile.xml and use the manifest to decode any custom event data.


Analyzing ETW files

ETW files can be read using the Windows API
The APIs that are available are documented in the Event Tracing Reference part of the MSDN library.

ETW and Debugging

ETW and Process Dumps
The ETW information is available in a dump. You can for example dump the content of the "Circular Kernel Session" to a file usign WinDbg. To do this issue the following command.
!wmitrace.strdump
!wmitrace.logsave <session> <filename>

For more info: WPP Tracing with !wmitrace and the Debugger

Memory (heap) Problems
ETW lets you trace the memory allocation done within a process. Combining this with the ability to log stack frames (xperf -stackwalk) enables you to debug memory problems. More on this can be found here:

Part 3: ETW Methods of Tracing
TraceLog


Resources
Network Tracing in Windows 7
GPUView


Thursday, December 29, 2011

PerfMonitor primer

Perfmonitor is a ETW based tool for 'profiling' managed code, it can be downloaded from CodePlex. The December issue of MSDN magazine has a nice article to get you started: Performance Diagnostics of .NET Applications Using ETW. It works best on .NET 4.0 but some features also work on CLR2.x/.NET 3.5.

One of the great things about PerfMonitor (besides being a free download) is that it is easy to install on a target system (no installers just copy the exe). Just carry it along on your USB drive and within minutes your are ready to gather data which can help you solve a performance problem.

The tool has a build in manual, running the tool without a commandline parameters will start your browser and will show the user guide.

There are two steps to using the tool
1. Collect data
2. Post processing of the data and generating a report.

A typical scenario might look like this:
  1. Get the process ID of the process your are monitoring (tasklist)
  2. start the collection: perfmonitor.exe collect
  3. <perform use case>
  4. <stop collection>
  5. Merge the ETL files that have been generard: performonitor.exe merge
  6. Generate the report: performonitor.exe /process:<ProcID> analyze 
  7. Evaluate the report
A list of command line options and qualifiers can be found below.

Data collection
To collect the data there are a number of options (the complete overview can be found in the table below):
  1. Explicitly start and stop the data collection (perfmonitor.exe start [<filename.etl>] / performonitor.exe stop)
  2. perfmonitor.exe collect
  3. perfmonitor.exe. runAnalyze <executable>
    The 'runAnalyze' option collects the data and automatically generates a report.

Data CollectionDescription
runAnalyze CommandAndArgs ...Performs a run command then the analyze command.
run CommandAndArgs ...Starts data collection, runs a command and stops.
collect [DataFile]Starts data logging, then displays a messagebox. When dismissed, it then stops collection.
start [DataFile]Starts machine wide profile data collection.
stopStop collecting profile data (machine wide).
abortInsures that any active PerfMonitor sessions are stopped.
listSessionsLists active ETW sessions.
Listen Providers ...Turns on ETW logging for EventSources (providers), and then writes text file (by default to stdout).
runDump CommandAndArgs ...Performs a run command then the Dump command.
monitor CommandAndArgs ...Turns on all event sources, performs a run command.
monitorDump CommandAndArgs ...Turns on all event sources, performs a run command then the Dump command.
monitorPrint CommandAndArgs ...Turns on all event sources, performs a run command then the PrintSources command.

Reporting
 After the data has been collected a report can be generated. This can be done using the following command: perfmonitor analyze [<filename.etl>]

There are several options that can be used when generating the report, here is a list of all the options.

ReportingDescription
analyzeCreates a general performance report report (CPU, GC, JIT ...).
merge [DataFile]Combine separate ETL files into a single ETL file (that can be decoded on another machine).
stats [DataFile]Produce an XML report of what events are in an ETL file.
procs [DataFile]Display the processes that started in the trace.
GCTimeCreates a Report on .NET garbage collection
CpuTimeCreates a Report on CPU usage time.
JitTimeCreates a Report on .NET Just In Time compilation.
etlx [DataFile]Create a ETLX file from the ETL files.
PrintSourcesTakes an ETL file and generates a text file that prints EventSource events as formatted strings.
dump [DataFile]Convert the events to an XML file.
rawDump [DataFile]Convert the events with no preprocessing (thus stacks are individual events, and rundown events are shown.

MiscDescription
UsersGuideDisplays the users Guide.
cpuTestRun a simple, CPU bound routine. Useful as a tutorial example.
listSources ExeFileLists all System.Diagnostics.EventSources in a executable (or its static dependencies)
monitorProcsEXPERIMENTAL: Monitor process creation in real time.


Here is an overview of all the qualifiers
Qualifiers Description
-BufferSize:INT32(64)The size the buffers (in MB) the OS should use. Increase this if PerfMonitor warns you that the log lost events.
-Circular:INT32(0)Do Circular logging with a file size in MB. Zero means non-circular.
-MergeDo a merge after stopping collection.
-EtlxConvert to etlx (contains symbolic information) after stopping collection.
-NoRundownDon't request CLR Rundown events.
-RundownTimeout:INT32(30)Maximum number of seconds to wait for CLR rundown to complete.
-MinRundownTime:INT32(0)Minimum number of seconds to wait for CLR rundown to complete.
-SymbolsForDlls:STRING,...A comma separated list of DLL names (without extension) for which to look up symbolic information (PDBs).
-PrimeSymbolsNormally only machine local paths are searched for symbol info. This indicates symbol servers should be searched too.
-DataFile:STRINGFileName of the profile data to generate.
-AdditionalProviders:STRING,...Providers in ADDITION to the kernel and CLR providers. This is comma separated list of ProviderGuid:Keywords:Level specs.
-Providers:STRING,...Providers to turn on INSTEAD of the kernel and CLR providers. This is comma separated list of Provider:Keywords:Level specs were Provider is either a GUID, or @FileName#EventSourceName.
-ClrEvents:KEYWORDS(Default)A comma separated list of .NET CLR events to turn on. Legal values: None, GC, Binder, Loader, Jit, NGen, StartEnumeration, StopEnumeration, Security, AppDomainResourceManagement, JitTracing, Interop, Contention, Exception, Threading, Default, Stack.
-KernelEvents:KEYWORDS(Default)A comma separated list of windows OS kernel events to turn on. See Users guide for details. Legal values: None, Process, Thread, ImageLoad, ProcessCounters, ContextSwitch, DeferedProcedureCalls, Interrupt, SystemCall, DiskIO, DiskFileIO, DiskIOInit, Dispatcher, MemoryPageFaults, MemoryHardFaults, VirtualAlloc, NetworkTCPIP, Registry, AdvancedLocalProcedureCalls, SplitIO, Driver, OS, Profile, Default, FileIO, FileIOInit, Verbose, All.
-NoBrowserDon't launch a browser on an HTML report.
-Process:STRINGFilter events to just one process with the given name (exe without extension) or numeric process ID.
-StartTime:DOUBLE(0)Filter events happening before this time (msec from start of trace.
-EndTime:DOUBLE(Infinity)Filter events happening after this time (msec from start of trace.
-Threshold:SINGLE(5)Stack items less than this % threshold are folded into their parent node.
-NoOSGroupingTurn off the grouping of OS functions in stack traces.
-NoBrowserDon't launch a browser on an HTML report.
-WriteXmlWrite an XML report as well as the HTML report.

Ok, I hope that this gets you started with this great tool!

Debugging Resources

I regularly surf the web to read about debugging tips and tricks. This entry contains a list resources that I found worth wile. I will be updating as I go along.

WinDbg

Visual Studio

Friday, December 23, 2011

A simple approach that paid off

I am currently migrating a relatively large application from VS2008/NET3.5 to VS2010/NET4. During this migration we ran into a problem. After we had migrated the code we wanted to test the application. For this we have a number of automated tests. We use Ranorex to drive part of our application.
While testing we found that the application reacted in a sluggish way and the automated tests failed due to time-outs.  We also observed that when doing the same tests ,manually the problem did occur. So what was happening and why this strange behavior.

I followed a simple approach  which paid of and which gave us enough information pin point the problem.
I fired up WinDbg attached to the process and loaded the sos extension (.loadby sos clr).
I listed all the thread (!threads) and looked up the thread ID of the UI thread.
I then resumed the program. 

I prepared the following command (17 being the ID of the UI thread):  ~17s;!clrstack;g

I then reproduced the problem. At that point I quickly hit break and executed the above command. 
I repeated this a number of times. After doing so I started looking at the different calls stacks.
What I noticed that all the stacks looked similar.

OS Thread Id: 0x1f08 (17)
Child SP IP       Call Site
0a54be30 5c075e00 Ranorex.Libs.WinForms.AssemblyResolver.AssemblyResolveHandler(System.Object, System.ResolveEventArgs)
0a54be34 68db0de1 System.AppDomain.OnAssemblyResolveEvent(System.Reflection.RuntimeAssembly, System.String)
0a54c074 6a2121db [GCFrame: 0a54c074] 
0a54d0ac 6a2121db [HelperMethodFrame_PROTECTOBJ: 0a54d0ac] System.Reflection.RuntimeAssembly._nLoad(System.Reflection.AssemblyName, System.String, System.Security.Policy.Evidence, System.Reflection.RuntimeAssembly, System.Threading.StackCrawlMark ByRef, Boolean, Boolean, Boolean)
0a54d154 68dce713 System.Reflection.RuntimeAssembly.InternalGetSatelliteAssembly(System.String, System.Globalization.CultureInfo, System.Version, Boolean, System.Threading.StackCrawlMark ByRef)
0a54d17c 68dce4ea System.Resources.ManifestBasedResourceGroveler.GetSatelliteAssembly(System.Globalization.CultureInfo, System.Threading.StackCrawlMark ByRef)
0a54d1c0 68dbd314 System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(System.Globalization.CultureInfo, System.Collections.Generic.Dictionary`2<System.String,System.Resources.ResourceSet>, Boolean, Boolean, System.Threading.StackCrawlMark ByRef)
0a54d230 68dbcf95 System.Resources.ResourceManager.InternalGetResourceSet(System.Globalization.CultureInfo, Boolean, Boolean, System.Threading.StackCrawlMark ByRef)
0a54d298 68dbcd51 System.Resources.ResourceManager.InternalGetResourceSet(System.Globalization.CultureInfo, Boolean, Boolean)
0a54d2b4 68dd324a System.Resources.ResourceManager.GetResourceSet(System.Globalization.CultureInfo, Boolean, Boolean)
.....

To further investigate the problem I put a breakpoint on the AssemblyResolveHandler. To get the address of the JIT'ed code I used the following command:

0:083> !name2ee Ranorex.Libs.WinForms.dll Ranorex.Libs.WinForms.AssemblyResolver.AssemblyResolveHandler
Module:      5c021000
Assembly:    Ranorex.Libs.WinForms.dll
Token:       060001cd
MethodDesc:  5c0354a4
Name:        Ranorex.Libs.WinForms.AssemblyResolver.AssemblyResolveHandler(System.Object, System.ResolveEventArgs)
JITTED Code Address: 5c075e00

Next I set a breakpoint using : bu 5c075e00 "!clrstack;g"
and opened a logfile (.logopen <filename>) and reran the test.

This way I found that there were indeed a large number of calls to this method. I repeated the experiment on our VS2008 build and here I did not see a large amount of calls to this method. 

The next step in the investigation was to examine the call-stack in more detail. We found that the "System.AppDomain.OnAssemblyResolveEvent" behave differently in framework version 4. MSDN states: 

Beginning with the .NET Framework 4, the ResolveEventHandler event is raised for all assemblies, including resource assemblies. In earlier versions, the event was not raised for resource assemblies. If the operating system is localized, the handler might be called multiple times: once for each culture in the fallback chain.

Debugging along we found that the library that our application is using could not find a resource (tooltip) which resulted in lots of callback to the Ranorex code. 

So using this simple method and the help of google (leading us to MSDN) we were able to pin point the problem in a relatively short time. 

Happy bug hunting

Sunday, December 18, 2011

The right tool for the job

Having the right tools for the job can save your day. Over the years I have been using a number of tools when trouble shooting problems. Here my shopping list of (mostly free) tools that I use in my day to day work.

If you have a favourite must have tool for trouble shooting which is not on the list then please let me know!

Performance

  • Xperf
  • ProcessExplorer: Taks manager on steroids
  • ProcMon: SysInternals tool for monitoring file, registry and network events
  • PerformanceMonitor: including PAL
  • PerfMonitor: .NET/CLR profiling
  • RML: SQL performance problems
  • AQ Time (commercial): profiling C# code
  • V-Tune (commercial)

Debugging

General Purpose Tools