To create a new repository into an existing directory (project) you have to run the init dialog. From the explorer context menu select TortoiseHg... ‣ Create Repository Here over the directory, or, within the folder, type hgtk init.
Repository Init Dialog
Creating a new repository means create a subdirectory called .hg. In this subdirectory Mercurial keeps all its versioning information.
Warning
It is dangerous to manually edit the files in .hg directory, repository corruption can occur. .hg/hgrc is perhaps the only exception to this rule.
The init tool can be started from command line
hgtk init
The syntax is
hgtk init [DEST]
where [DEST] is the path to destination folder.