Fritzing likes the board svg to be very specific. Since you created it in Freecad you will need to break apart the svg and then do a difference to create the holes. To do this in inkscape select all (ctrl a) and then break apart (ctrl shift k). Now select just the parts you want as holes while holding shift (you may have to be in node editing F2). Then merge them with a union (ctrl +). Now you should have two parts. The board and the holes. Now select both of them while holding shift and do a difference (ctrl -) which will cut out the holes from the board. You must do the difference last. If you move any vertices after the difference you have to break it apart and do the unions and difference again as the order the vertices are defined in the file matters and Fritzing will not be able to make the hole in the contour file.
You also need to have the group ids set correctly including case.
board not Board
silkscreen not Silkscreen
silkscreen0 not Silkscreen0
Board and silkscreen color does not actually seem to matter at all.
See here for my Inkscape tips and tricks. http://forum.fritzing.org/t/inkscape-tips-and-tricks/