I think the graph below more or less tells the story.
There is absolutely no performance difference regarding which Writer you choose. That's the short story anyway.
The longer story goes something like this...
According to the JDK 7 documentation for StringWriter, it is backed by a StringBuffer, which is thread-safe. However the CharArrayWriter documentation says it is backed by a simple char array. This may lead one to think that CharArrayWriter is not thread safe. That's what I thought as there is no mention of it otherwise.
You'd be wrong if you thought that CharArrayWriter is not thread safe. Examining the code reveals that all the critical parts of the code are wrapped with a synchronized block. This makes it equivalent to StringWriter's thread-safe StringBuffer. CharArrayWriter is thread safe.
I did investigate a little bit more and even wrote some code that used a single CharArrayWriter between multiple threads to see if I could produce inconsistent appends, this didn't not happen in any of my tests. I used a regex pattern to test my generated string after every test.
This is the code I used to test. Swap the StringWriter/CharArrayWriter as needed.
The only discernible difference I could see is with CharArrayWriter you can get access to the backing char array (or copy of it at least) directly whereas its a little bit more work to get the char array when using StringWriter.
I was a little surprised that CharArrayWriter is thread safe and would like to see an unsynchronized implementation of a string based writer in the JDK in the future.