I know of no way to kill caterpillars without using chemicals or killing other 'healthy' garden bugs. If there is, I hope someone will share their 'tip'.
This is what I found in a gardening book I have titled, "The Veggie Gardener's Answer Book" (100% Organic).
Insects that prey on caterpillars and grubs include yellow jackets, hornets, ground beetles, tachinid flies, and many true bugs.
That's all I have to offer. I wish my garden was in and growing!!! We have had a LATE, cold, wet spring.
Water