Semi-recently, I attended a workshop-style conference. While I think I really like the organization and their work, there were some issues in organizing around "techies". So from that, I'd like to present some suggestions as to how to run one of these events:
Make sure your wireless works. If your tooling lives on GitHub, and your venue's wireless blocks GitHub, you're going to have a bad time. (This is not a hypothetical example.)
Be sure you have things to be done. This requires effort on your part: not only checking that the work exists, but that it is of reasonable scope.
Match work to participants. In particular, do not give programming jobs to people who have never programmed before. There is a tendency to be welcoming; keep in mind that people who are over their head are unlikely to return.
Don't sweat inefficiency. Someone who knows the project or codebase already will be able to accomplish the tasks you have prepared faster (and possible better) than your attendees. If the primary purpose of the event is either (or both!) of onboarding or face-to-face collaboration, then there's no reason to worry.
Have mentors. Your participants will want to excel regardless of the previous item; it's natural. And they'll feel better if they are enabled to do so; having a better time makes people more likely to stick around.
Give time for schmoozing. It's best if this isn't explicitly labeled in the schedule. Ideally, it's slop around moving from place to place, or longer-than-necessary breaks for food, and things like that. This way people do not feel forced to "network", but are still free to socialize.
Formally collect all work at the end. You of course care some about the work being done (hopefully), so it makes sense to gather it. There is also value in associating people with a project since then they feel accountable for it, and are more likely to continue working on it in the future.
Contextualize. Make sure your participants know why the work they're doing is important, and what more they can do once they have good knowledge of the project.
I don't mean this list to be exhaustive: while not sufficient for a good conference, these are I judge necessary conditions. I think these events can be very valuable, especially for bringing on new people, and don't want to discourage them in any way, just improve them.