Eh, it really should be a separate repository IMO, as it makes it easier for people to get involved, contribute, and for you to generate tarballs and tag releases if needed. If you need to pull it in to your other repo for some reason, that is what git submodules are for.