Out and About: November 14th, Inland Empire, CA

While things have been slow on my blog...they have not been slow in my life. I’m off next week to San Bernardino, CA to talk about concurrency, threads, and grid computing. If you missed my talk at the San Gabriel Valley .NET user group, here’s your chance to catch it with the Inland Empire .NET User’s Group. You should register so that the organizers can plan ahead.

This will be my first time to San Bernardino, although the county figured very largely in my life a few years ago. My youngest son is adopted out of the foster care system. But he is the second child that we had been presented with. The first child was living in San Bernardino. After evaluating his medical needs, we had decided that we were not going to be a good fit. He has a heart condition which requires good temperature control in the home and no stairs. Our house is a converted cabin with no insulation and the front door of the house is 36 steps up from the street. Anyone who can read knows that the housing prices in the Bay Area are crazy, so moving wasn’t an option. We always wonder about James. How would our lives be different if he had joined our family? How would his life have been different? Who is his family now? How is he doing?

So, I’ll be looking around and thinking about him. And I am looking forward to talking to the group!

Out and About: October 18th, San Gabriel Valley, CA

Wow! I’ve been writing a ton of articles...unfortunately I’ve been so busy they haven’t made it to the blog. I’m off to Monrovia, CA tomorrow to talk to the San Gabriel Valley .NET User Group about concurrency.

When I had started talking to the Digipede folks about joining their team I hadn’t given a lot of thought to understanding what I love about software development. Over the last year I have found myself evaluating my career and thinking back on the products and projects I loved. From a technical perspective: I love the low level stuff, I love debugging, and I love concurrency. I also love to teach and coach so I’m finding that presenting on concurrency is a lot of fun. I also believe that helping people understand concurrency design issues and how threading works, and sharing hard-earned lessons improves the quality of products we deliver. My stepfather Gary always said, “Leave things better then you found them.” That is always my hope when I give one of these talks. That people have at least one take away from the talk that will help them be a better engineer.

So! I’ll be down south tomorrow. If you’re in the area join us. Make sure to register and bring $5 for the pizza.

Silicon Valley Code Camp: Wrap up

Well, I had a lot of fun on Sunday with both of my presentations; both were well attended, and in both I had some really good questions and contributions from the attendees. One of the things that I was very surprised to see was the number of women at the event. I’ve been in this industry since 1987 and I had grown accustomed to being very outnumbered by the men. Many times, I was the only woman. Not on Sunday. It’s nice to see the industry changing. And perhaps this change will mean that being successful in this industry as a woman, doesn’t require that a woman be 4x better than all the men.

For all of you who have submitted session reviews: Thank you. Detailed reviews are valuable tools to help improve a presentation. However, some of the reviews pointed out a problem that I also see reflected in the world of blogging. People don’t read. Or maybe I made too many assumptions about their comprehension of what I wrote. In the future I’ll try to provide a clearer title. The reviews for my debugging session went from really liked the session to the session was useless. What’s interesting to me is that the people who didn’t like the session didn’t seem to understand what I was planning to talk about. I specifically stated in the session description that we would be looking at the disassembly, register, and memory windows. Those windows have little value if we don’t talk about Assembly. A lot of debugging is learning to creatively use the tools that you have available to you. If you don’t understand the information the tools expose then you won’t be able to do that.

I think that both debugging and performance are two huge subjects that we could do more to talk about and, as I suggested in the debugging session, I think we would all benefit from targeted performance and debugging conversations.

I also realized that I really don’t know what the definition for Beginner, Intermediate, and Advanced are. Especially in technology. While I think it is easier to assess a person’s level as an athletic coach, determining the level for a technology presentation is harder. Another factor that a lot of people don’t realize is that in any presentation each attendee brings unique skills and experiences to the session. This makes it hard for a presenter to put together a generic presentation that will benefit everyone. Next code camp, I’ll ask the organizers if they can post some standard level definitions so that we will all have a better idea what to expect.

For those of you who wanted more in the area of grid computing, in particular Digipede...Well, out of respect for the code camp I tried to stay away from a sales type of presentation. I encourage you to go to this webpage and request a free copy of the Digipede Network Developer Edition. Yes, you do have to fill out some contact information, but really it’s not that big a deal. The Digipede Network Developer Edition is a fully featured version of the Digipede Network Team Edition. The only limitation is that your grid can only install two Agents. But you get everything. The Digipede Server, Digipede Agent, Digipede Control (administration tool), Digipede Workbench (GUI app that creates and runs Jobs via a wizard), Digipede Framework SDK (this is the cool stuff), and complete documentation with code samples. And if you were looking for more information you can also cruise around the main site. We’ve put up a bunch of videos and some whitepapers to try and make it easier for people to understand the technology.

Many thanks to the code camp organizers and to all the volunteers and donors who made the event such a success.

Show me the money!

The other evening I was talking with some software folks about technology and software development. The topic ranged over to which language is better. For those of you who have been reading my blog for awhile you know that I think these types of conversations are ignorant. Before it got too heated, I just had to break in and say, "It’s about the money." That stopped the conversation track cold because all the participants were senior level, experienced software development folk. And they know it’s all about the money. If someone pays you to write code for them...then every technology choice is some how tied to money. If it is a commercial
application then the software needs to generate revenue to pay your salary. If it is an internal application then it should save the company money. The only time that software development is only about the technology is when you write it for free.

Understanding the business impact of your technology choices is one of the differences between being an architect and being a coder. An architect will look at the problems and make technology choices that are balanced between short-term objectives (time to market, stability, performance, scalability, security, ...) and long-term objectives (maintainability, enhance ability, marketability, ...). Developers who make pure technology choices (i.e. without considering the business aspects of the choice) are far more likely to build a product that will fail.

Over the years I’ve had the opportunity to work for quite a lot of startups. Startups that were founded by pure technologists tended to fail because the leaders didn’t understand marketing and sales. But not just marketing and sales from the perspective of what the marketing and sales teams needed to do, but in terms of what the technology needed to do to satisfy a market. Any market.

I have also worked for companies started by sales folks who understood the sales-side of things but didn’t understand the technology. These companies are interesting because they jumped right out of the gate but the technology wasn’t solid enough to standup to customer use. (Just for the record, I often come in late on companies.)

Companies with the highest success rates were those where the founding teams were balanced and could communicate between the technology and sales sides. Building high quality products, using the right technologies for the problem space, in a market that needs the solution.

Just remember the old adage, "If all you have is a hammer, then everything is a nail." And keep in mind what that means. These days the software community has a wide assortment of tools to choose from. There is a smorgasbord of choices. We have today the equivalent of my father’s SnapOn tool chest. Use it.

Digipede and Dr. Dobbs

Robert and Dan wrote an article for Dr. Dobbs a few months ago and it was published today. The article is Scaling SOA with Distributed Computing. I just finished reading it and I think the guys did a great job. I encourage you to check it out.

Links: October 5, 2006

Concurrency/Performance
UI Controls and Threading
An Introduction to programming with C# Threads
Programming with C# Threads Presentation
Do I Thread Thee? Let Me Count The Ways...
What Every Dev Must Know About Multithreaded Apps
Processes, threads and synchronization
Threading in C#
On Efficiency, Scalability, and the Wisdom to Know the Difference
Multicore Standards
Quad Core
Asynchronous Tasks
Understand problem before you try to find a solution
To Grid Or Not To Grid?
Clustering in VMWare 1.0
Surf's Up!
Why Processes Scale Better Than Threads
Code Analysis Tools

Debugging
dotTrace 2.0 beta
Simpler Debugger with DebuggerTypeProxy
Custom service controller for debugging windows services
Deep Troubleshooting Tools for Windows - Kernel Memory Space Analyzer Version 8.1
Debugging Winsock LSPs
Resources for Debugging .NET
Debugging Ngen code
Debug a process which is attached to another Debugger
Debugging 100

Embedded/Mobile
Blogging about embedded software development
Mobile Trends and Technologies
Windows Mobile Labs
Windows XP Embedded Virtual Labs
Understanding Memory Sections in config.bib, boot.bib, and OEMAddressTable in Windows CE 5.0 and 6.0
Component Enhancements in Windows XP Embedded SP2 Feature Pack 2007

SOA/SaaS
A Service is Not an Object or Component
Responses to SOA and the Reality of Reuse
More SOA tips from the real world
Patterns and Anti-Patterns for SOA
Microsoft: Real-world SOA
Understanding SaaS Architecture Powerpoint Presentation

Learning
Free Computer Books, Tutorials & Lecture Notes
Intro to Cryptography Course
High level white papers on .Net framework 3.0
Patterns and Practices Guidance Explorer
Podcasts I listen to - level 100
Free Books, Journals, & Articles on Computer Science, Engineering, Programming, & Math
Free MS Courses and Workshops

Just Fun
My foray into XNA : Basic Pixel Perfect Collision Detection
Write your own Code Generator or Template Engine in .NET
A Comparison of the Linux and Windows Device Driver Architectures
Key Driver Concepts and Driver Development books

Out and About: October 8th, Silicon Valley Code Camp

This Sunday I’m giving two talks at the Silicon Valley Code Camp in Los Altos Hills, CA. Foothill College is hosting the event and from the pictures it looks like they have a nice campus. Registration is still open and the event is free, so if you're in the San Francisco Bay Area...come on down.

Dan is also presenting at this event and you can catch his session titled ".NET Development and Excel Services" at 9:15 on Sunday morning. Dan has been working with Excel Services for several months now and did an MSDN webcast on the subject this morning. You will soon be able to watch that webcast here. Dan also put up a post for more information on Excel Services here.

Sunday at 1:15 PM Concurrent Software Development

** this is the blurb from the session wiki **

My talk on concurrent software development was designed for a two hour session. Because the code camp format is one hour I’m dropping most of the high level discussion about why concurrency matters and concurrency design. Instead you will find some of my thoughts here as well as links to references where you can learn more. I will also continue to blog on concurrency so I encourage you to visit from time to time. (http://krgreenlee.blogspot.com)

Why Concurrency Matters

Concurrent software development is not new. If you’ve written code that uses threads then you’ve written code that, in theory, is concurrent. However, with computers that have only one CPU, true concurrency does not happen. The opportunity for real concurrency is increasing. Recently the hardware vendors have started making multi-core machines. A dual-core laptop is now on the market and affordably priced. This puts multi-core machines into the hands of the average user. Previously, the expense of multi-processor boxes had relegated those machines to the server room. No more.

The reason for the shift to multi-core machines is that the faster chips are generating too much heat. Until a cooling solution is found we are not going to see any faster chips. To get around this problem the hardware developers have started adding cores (usually with slower chips) and increasing the on chip cache size.

In the past when hardware developers came out with a faster chip, software automatically got faster. This was great for both developers and users. But with multi-core machines this is not the case. For the software to run faster, the software companies are going to have to modify their software to take advantage of the available cores. Software that is already concurrent, but never tested on multi-processor machines, may fail.

For a long time software developers could count on the hardware getting faster so an emphasis on performance, for many products, didn’t exist. I’ve even had conversations with engineers who told me to stop worrying about the efficiency of an algorithm because of that very fact. Well folks, responsibility for performance improvement has landed squarely back into our laps. We, the software development community, have work to do.

"The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software" – Herb Sutter

Concurrency Design Considerations

You've decided that you need to add concurrency to your applications. The questions then becomes, "Where?" and "How?" Finding those areas in your applications that would benefit from concurrency is called decomposition. There is data decomposition and functional decomposition. And sometimes there is overlap between them. Identifying opportunities for performance improvement is an ongoing process and that is the same for concurrency opportunities. As your understanding of concurrency improves so will your ability to identify those opportunities.

Let's say that you have an application that processes very large EDI files. These files come in at the end of each month and it's a mad house trying to get the files processed quickly. Right now the application is fed a file and runs for six hours. How can you use concurrency to speed that up? Programmatically what is happening is an ETL (extract, transform, and load) process. Because you know that there are transaction boundaries within the EDI file, you can easily break the large EDI file up into smaller EDI files. This is where your opportunity to add concurrency occurs. By breaking up the large file into much smaller ones, the application can then process each of the smaller files concurrently. Significantly reducing the processing time.

Or let's say you have an application that does risk analysis. You are running a lot of Monte Carlo simulations to attempt to evaluate the risk potential of an investment. The application execution takes eight hours. Because a Monte Carlo simulation is based on random number generation it is an algorithm that is very parallelizable. With concurrency you can significantly reduce the execution time or if the execution time is fine you could change the algorithm to run more simulations in an attempt to get a more accurate result.

Some other areas to look at during decomposition are places where the application interfaces with a slow device or a human being. A slow device may be the hard drive, and you may want one execution path that reads/writes to the hard drive while another execution path processes the data. I think one of the most common places to find parallel execution paths is between the UI (foreground thread) and a background thread. Giving the user the impression that an application is responsive is an important contributing factor to how the average user perceives the value of a product.

Another area to look for concurrency opportunities is in loops. Long running loops, where loop iterations do not have dependencies on past iterations, are great candidates.

The other two concurrency design considerations are synchronization and communication. These two areas are tool dependent. Meaning that the language and tools you use to write your application directly affect your synchronization and communication options.

The above is a quick summary of the first half of my two hour session. The rest is code. However, if you want to read some good article on concurrency and threading I suggest you read the following:

"Using concurrency for scalability" – Joe Duffy
"What Every Dev Must Know About Multithreaded Apps" – Vance Morrison

Sunday 10:45 AM VS2005: Debugging Tips and Tricks

I want this debugging session to be an opportunity for sharing information. I'm planning on looking at the different debug tools. I’ll share some tips and tricks I've learned over the years and I hope that others in attendance will share theirs.

See you on Sunday.