更新 README.md

Merge pull request !2 from hfu/master

README.md

@@ -1,14 +1,14 @@
 <div align=center>
-<img src="http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/dface_logo.png" width="350">
-</div>
+<a href="http://dface.tech" target="_blank"><img src="http://dftech.oss-cn-hangzhou.aliyuncs.com/web/DFACE-logo_dark.png" width="350"></a>
+</div>  
 
 -----------------
-# DFace • [![License](http://pic.dface.io/apache2.svg)](https://opensource.org/licenses/Apache-2.0)
+# DFace (Deeplearning Face) • [![License](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/apache_2.svg)](https://opensource.org/licenses/Apache-2.0)
 
 
 | **`Linux CPU`** | **`Linux GPU`** | **`Mac OS CPU`** | **`Windows CPU`** |
 |-----------------|---------------------|------------------|-------------------|
-| [![Build Status](http://pic.dface.io/pass.svg)](http://pic.dface.io/pass.svg) | [![Build Status](http://pic.dface.io/pass.svg)](http://pic.dface.io/pass.svg) | [![Build Status](http://pic.dface.io/pass.svg)](http://pic.dface.io/pass.svg) | [![Build Status](http://pic.dface.io/pass.svg)](http://pic.dface.io/pass.svg) |
+| [![Build Status](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg)](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg) | [![Build Status](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg)](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg) | [![Build Status](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg)](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg) | [![Build Status](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg)](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/build_pass.svg) |
 
 
 **基于多任务卷积网络(MTCNN)和Center-Loss的多人实时人脸检测和人脸识别系统。**
@@ -27,10 +27,12 @@ DFace可以利用CUDA来支持GPU加速模式。我们建议尝试linux GPU这
 
 **MTCNN 结构**　　
 
-![mtcnn](http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/mtcnn_st.png)
+![Pnet](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/pnet.jpg)
+![Rnet](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/rnet.jpg)
+![Onet](http://dftech.oss-cn-hangzhou.aliyuncs.com/opendface/img/onet.jpg)
 
 
-** 如果你对DFace感兴趣并且想参与到这个项目中, 以下TODO是一些需要实现的功能，我定期会更新，它会实时展示一些需要开发的清单。提交你的fork request,我会用issues来跟踪和反馈所有的问题。也可以加DFace的官方Q群 681403076 也可以加本人微信 jinkuaikuai005 **
+**如果你对DFace感兴趣并且想参与到这个项目中, 以下TODO是一些需要实现的功能，我定期会更新，它会实时展示一些需要开发的清单。提交你的fork request,我会用issues来跟踪和反馈所有的问题。也可以加DFace的官方Q群 681403076 也可以加本人微信 jinkuaikuai005**
 
 ###  TODO(需要开发的功能)
 - 基于center loss 或者triplet loss原理开发人脸对比功能，模型采用ResNet inception v2. 该功能能够比较两张人脸图片的相似性。具体可以参考 [Paper](https://arxiv.org/abs/1503.03832)和[FaceNet](https://github.com/davidsandberg/facenet)
@@ -41,7 +43,8 @@ DFace可以利用CUDA来支持GPU加速模式。我们建议尝试linux GPU这
 - Docker支持，gpu版
 
 ## 安装
-DFace主要有两大模块，人脸检测和人脸识别。我会提供所有模型训练和运行的详细步骤。你首先需要构建一个pytorch和cv2的python环境，我推荐使用Anaconda来设置一个独立的虚拟环境。
+DFace主要有两大模块，人脸检测和人脸识别。我会提供所有模型训练和运行的详细步骤。你首先需要构建一个pytorch和cv2的python环境，我推荐使用Anaconda来设置一个独立的虚拟环境。**如果使用GPU训练模式，需要安装Nvidia的cuda和cudnn。** 目前作者倾向于Linux Ubuntu安装环境。感谢热心网友提供windows DFace安装体验，windos安装教程具体
+可参考他的[博客](http://www.alearner.top/index.php/2017/12/23/dface-pytorch-win64-gpu)
 
 
 ### 依赖
@@ -52,89 +55,111 @@ DFace主要有两大模块，人脸检测和人脸识别。我会提供所有模
 * cv2
 * matplotlib
 
-在这里我提供了一个anaconda的环境依赖文件environment.yml，它能方便你构建自己的虚拟环境。
+```shell
+git clone https://gitee.com/kuaikuaikim/dface.git
+```
+
+在这里我提供了一个anaconda的环境依赖文件environment.yml （windows请用environment-win64.yml,Mac OSX请用environment_osx.yaml），它能方便你构建自己的虚拟环境。
+
+```shell
+cd dface  
+
+conda env create -f environment.yml
+```
+
+添加python搜索模块路径  
 
 ```shell
-conda env create -f path/to/environment.yml
+export PYTHONPATH=$PYTHONPATH:{your local DFace root path}
 ```
 
+
+
 ### 人脸识别和检测
 
 如果你对mtcnn模型感兴趣，以下过程可能会帮助到你。
 
 #### 训练mtcnn模型
 
-MTCNN主要有三个网络，叫做**PNet**, **RNet** 和 **ONet**。因此我们的训练过程也需要分三步先后进行。为了更好的实现效果，当前被训练的网络都将依赖于上一个训练好的网络来生成数据。所有的人脸数据集都来自　**[WIDER FACE](http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/)** 和 **[CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)**。WIDER FACE仅提供了大量的人脸边框定位数据，而CelebA包含了人脸关键点定位数据。
+MTCNN主要有三个网络，叫做**PNet**, **RNet** 和 **ONet**。因此我们的训练过程也需要分三步先后进行。为了更好的实现效果，当前被训练的网络都将依赖于上一个训练好的网络来生成数据。所有的人脸数据集都来自　**[WIDER FACE](http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/)** 和 **[CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)**。WIDER FACE仅提供了大量的人脸边框定位数据，而CelebA包含了人脸关键点定位数据。以下训练除了　生成ONet的人脸关键点训练数据和标注文件　该步骤使用CelebA数据集，其他一律使用WIDER FACE。如果使用wider face的 wider_face_train.mat 注解文件需要转换成txt格式的，我这里用h5py写了个 [转换脚本](https://gitee.com/kuaikuaikim/dface/blob/master/dface/prepare_data/widerface_annotation_gen/transform.py). 这里我提供一个已经转换好的wider face注解文件 [anno_store/wider_origin_anno.txt](https://gitee.com/kuaikuaikim/dface/blob/master/anno_store/wider_origin_anno.txt), 以下训练过程参数名--anno_file默认就是使用该转换好的注解文件。
+
+  
+* 创建 dface 训练数据临时目录，对应于以下所有的参数名　--dface_traindata_store
+```shell
+mkdir {your dface traindata folder}
+```
 
 
 * 生成PNet训练数据和标注文件
 
 ```shell
-python src/prepare_data/gen_Pnet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path}
+python dface/prepare_data/gen_Pnet_train_data.py --prefix_path {注解文件中图片的目录前缀,就是wider face图片所在目录} --dface_traindata_store {之前创建的dface训练数据临时目录}　--anno_file {wider face 注解文件,可以不填，默认使用anno_store/wider_origin_anno.txt}
 ```
 * 乱序合并标注文件
 
 ```shell
-python src/prepare_data/assemble_pnet_imglist.py
+python dface/prepare_data/assemble_pnet_imglist.py
 ```
 
 * 训练PNet模型
 
 
 ```shell
-python src/train_net/train_p_net.py
+python dface/train_net/train_p_net.py
 ```
 * 生成ＲNet训练数据和标注文件
 
 ```shell
-python src/prepare_data/gen_Rnet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path} --pmodel_file {yout PNet model file trained before}
+python dface/prepare_data/gen_Rnet_train_data.py --prefix_path {注解文件中图片的目录前缀，就是wider face图片所在目录} --dface_traindata_store {之前创建的dface训练数据临时目录} --anno_file {wider face 注解文件,可以不填，默认使用anno_store/wider_origin_anno.txt} --pmodel_file {之前训练的Pnet模型文件}
 ```
 * 乱序合并标注文件
 
 ```shell
-python src/prepare_data/assemble_rnet_imglist.py
+python dface/prepare_data/assemble_rnet_imglist.py
 ```
 
 * 训练RNet模型
 
 ```shell
-python src/train_net/train_r_net.py
+python dface/train_net/train_r_net.py
 ```
 
 * 生成ONet训练数据和标注文件
 
 ```shell
-python src/prepare_data/gen_Onet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path} --pmodel_file {yout PNet model file trained before} --rmodel_file {yout RNet model file trained before}
+python dface/prepare_data/gen_Onet_train_data.py --prefix_path {注解文件中图片的目录前缀，就是wider face图片所在目录} --dface_traindata_store {之前创建的dface训练数据临时目录} --anno_file {wider face 注解文件,可以不填，默认使用anno_store/wider_origin_anno.txt} --pmodel_file {之前训练的Pnet模型文件} --rmodel_file {之前训练的Rnet模型文件}
 ```
 
-* 生成ONet的人脸关键点训练数据和标注文件
+* 生成ONet的人脸五官关键点训练数据和标注文件
 
 ```shell
-python src/prepare_data/gen_landmark_48.py
+python dface/prepare_data/gen_landmark_48.py
 ```
 
-* 乱序合并标注文件(包括人脸关键点)
+* 乱序合并标注文件(包括人脸五官关键点)
 
 ```shell
-python src/prepare_data/assemble_onet_imglist.py
+python dface/prepare_data/assemble_onet_imglist.py
 ```
 
 * 训练ONet模型
 
 ```shell
-python src/train_net/train_o_net.py
+python dface/train_net/train_o_net.py
 ```
 
-#### 测试人脸检测
+#### 测试人脸检测  
+
+**如果不想训练，我已经把onet_epoch.pt,pnet_epoch.pt,rnet_epoch.pt三个文件放到model_store目录，直接运行test_image.py即可**
+
 ```shell
 python test_image.py
 ```    
 
 ### 人脸对比  
+TODO 根据center loss实现人脸识别
 
-@TODO 根据center loss实现人脸识别
-
-##　测试效果  
+#### 测试效果  
 ![mtcnn](http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/dface_demoall.PNG)  
 
 

dface/core/detect.py

@@ -1,12 +1,12 @@
+from __future__ import print_function
 import cv2
 import time
 import numpy as np
 import torch
 from torch.autograd.variable import Variable
-from models import PNet,RNet,ONet
-import utils as utils
-import image_tools
-
+from dface.core.models import PNet,RNet,ONet
+import dface.core.utils as utils
+import dface.core.image_tools as image_tools
 
 def create_mtcnn_net(p_model_path=None, r_model_path=None, o_model_path=None, use_cuda=True):
 
@@ -14,23 +14,30 @@ def create_mtcnn_net(p_model_path=None, r_model_path=None, o_model_path=None, us
 
     if p_model_path is not None:
         pnet = PNet(use_cuda=use_cuda)
-        pnet.load_state_dict(torch.load(p_model_path))
         if(use_cuda):
+            pnet.load_state_dict(torch.load(p_model_path))
             pnet.cuda()
+        else:
+            # forcing all GPU tensors to be in CPU while loading
+            pnet.load_state_dict(torch.load(p_model_path, map_location=lambda storage, loc: storage))
         pnet.eval()
 
     if r_model_path is not None:
         rnet = RNet(use_cuda=use_cuda)
-        rnet.load_state_dict(torch.load(r_model_path))
         if (use_cuda):
+            rnet.load_state_dict(torch.load(r_model_path))
             rnet.cuda()
+        else:
+            rnet.load_state_dict(torch.load(r_model_path, map_location=lambda storage, loc: storage))
         rnet.eval()
 
     if o_model_path is not None:
         onet = ONet(use_cuda=use_cuda)
-        onet.load_state_dict(torch.load(o_model_path))
         if (use_cuda):
+            onet.load_state_dict(torch.load(o_model_path))
             onet.cuda()
+        else:
+            onet.load_state_dict(torch.load(o_model_path, map_location=lambda storage, loc: storage))
         onet.eval()
 
     return pnet,rnet,onet
@@ -623,7 +630,7 @@ face candidates:%d, current batch_size:%d"%(num_boxes, batch_size)
 
             t3 = time.time() - t
             t = time.time()
-            print "time cost " + '{:.3f}'.format(t1+t2+t3) + '  pnet {:.3f}  rnet {:.3f}  onet {:.3f}'.format(t1, t2, t3)
+            print("time cost " + '{:.3f}'.format(t1+t2+t3) + '  pnet {:.3f}  rnet {:.3f}  onet {:.3f}'.format(t1, t2, t3))
 
         return boxes_align, landmark_align
 

dface/core/imagedb.py

@@ -1,6 +1,8 @@
+from __future__ import print_function
 import os
 import numpy as np
 
+
 class ImageDB(object):
     def __init__(self, image_annotation_file, prefix_path='', mode='train'):
         self.prefix_path = prefix_path
@@ -137,7 +139,7 @@ class ImageDB(object):
         imdb: dict
             image database with flipped image annotations added
         """
-        print 'append flipped images to imdb', len(imdb)
+        print('append flipped images to imdb', len(imdb))
         for i in range(len(imdb)):
             imdb_ = imdb[i]
             m_bbox = imdb_['bbox_target'].copy()

dface/core/resnet_inception_v2.py

@@ -256,6 +256,7 @@ class InceptionResnetV2(nn.Module):
         self.conv2d_7b = BasicConv2d(2080, 1536, kernel_size=1, stride=1)
         self.avgpool_1a = nn.AvgPool2d(8, count_include_pad=False)
         self.classif = nn.Linear(1536, num_classes)
+        self.dropout = nn.Dropout(p=0.8)
 
     def forward(self, x):
         x = self.conv2d_1a(x)
@@ -275,5 +276,6 @@ class InceptionResnetV2(nn.Module):
         x = self.conv2d_7b(x)
         x = self.avgpool_1a(x)
         x = x.view(x.size(0), -1)
+        x = self.dropout(x)
         x = self.classif(x)
         return x

dface/prepare_data/assemble_onet_imglist.py

@@ -1,6 +1,7 @@
+from __future__ import print_function
 import os
-import config
-import assemble as assemble
+import dface.config as config
+import dface.prepare_data.assemble as assemble
 
 
 if __name__ == '__main__':
@@ -22,4 +23,4 @@ if __name__ == '__main__':
     imglist_file = os.path.join(anno_dir, imglist_filename)
 
     chose_count = assemble.assemble_data(imglist_file ,anno_list)
-    print "PNet train annotation result file path:%s" % imglist_file
+    print("PNet train annotation result file path:%s" % imglist_file)

dface/prepare_data/assemble_pnet_imglist.py

@@ -1,6 +1,7 @@
+from __future__ import print_function
 import os
-import config
-import assemble as assemble
+import dface.config as config
+import dface.prepare_data.assemble as assemble
 
 
 if __name__ == '__main__':
@@ -22,4 +23,4 @@ if __name__ == '__main__':
     imglist_file = os.path.join(anno_dir, imglist_filename)
 
     chose_count = assemble.assemble_data(imglist_file ,anno_list)
-    print "PNet train annotation result file path:%s" % imglist_file
+    print("PNet train annotation result file path:%s" % imglist_file)

dface/prepare_data/assemble_rnet_imglist.py

@@ -1,6 +1,7 @@
+from __future__ import print_function
 import os
-import config
-import assemble as assemble
+import dface.config as config
+import dface.prepare_data.assemble as assemble
 
 
 if __name__ == '__main__':
@@ -22,4 +23,4 @@ if __name__ == '__main__':
     imglist_file = os.path.join(anno_dir, imglist_filename)
 
     chose_count = assemble.assemble_data(imglist_file ,anno_list)
-    print "PNet train annotation result file path:%s" % imglist_file
+    print("PNet train annotation result file path:%s" % imglist_file)

dface/prepare_data/gen_Onet_train_data.py

@@ -1,16 +1,18 @@
+from __future__ import print_function
 import argparse
-
 import cv2
 import numpy as np
-from core.detect import MtcnnDetector,create_mtcnn_net
-from core.imagedb import ImageDB
-from core.image_reader import TestImageLoader
+from dface.core.detect import MtcnnDetector,create_mtcnn_net
+from dface.core.imagedb import ImageDB
+from dface.core.image_reader import TestImageLoader
 import time
 import os
 import cPickle
-from core.utils import convert_to_square,IoU
-import config
-import core.vision as vision
+from dface.core.utils import convert_to_square,IoU
+import dface.config as config
+import dface.core.vision as vision
+
+
 
 def gen_onet_data(data_dir, anno_file, pnet_model_file, rnet_model_file, prefix_path='', use_cuda=True, vis=False):
 
@@ -27,7 +29,7 @@ def gen_onet_data(data_dir, anno_file, pnet_model_file, rnet_model_file, prefix_
 
     for databatch in image_reader:
         if batch_idx % 100 == 0:
-            print "%d images done" % batch_idx
+            print("%d images done" % batch_idx)
         im = databatch
 
         t = time.time()
@@ -59,14 +61,14 @@ def gen_onet_data(data_dir, anno_file, pnet_model_file, rnet_model_file, prefix_
         cPickle.dump(all_boxes, f, cPickle.HIGHEST_PROTOCOL)
 
 
-    gen_onet_sample_data(data_dir,anno_file,save_file)
+    gen_onet_sample_data(data_dir,anno_file,save_file,prefix_path)
 
 
 
 
 
 
-def gen_onet_sample_data(data_dir,anno_file,det_boxs_file):
+def gen_onet_sample_data(data_dir,anno_file,det_boxs_file,prefix):
 
     neg_save_dir = os.path.join(data_dir, "48/negative")
     pos_save_dir = os.path.join(data_dir, "48/positive")
@@ -89,11 +91,11 @@ def gen_onet_sample_data(data_dir,anno_file,det_boxs_file):
     im_idx_list = list()
     gt_boxes_list = list()
     num_of_images = len(annotations)
-    print "processing %d images in total" % num_of_images
+    print("processing %d images in total" % num_of_images)
 
     for annotation in annotations:
         annotation = annotation.strip().split(' ')
-        im_idx = annotation[0]
+        im_idx = os.path.join(prefix,annotation[0])
 
         boxes = map(float, annotation[1:])
         boxes = np.array(boxes, dtype=np.float32).reshape(-1, 4)
@@ -112,7 +114,7 @@ def gen_onet_sample_data(data_dir,anno_file,det_boxs_file):
     det_handle = open(det_boxs_file, 'r')
 
     det_boxes = cPickle.load(det_handle)
-    print len(det_boxes), num_of_images
+    print(len(det_boxes), num_of_images)
     assert len(det_boxes) == num_of_images, "incorrect detections or ground truths"
 
     # index of neg, pos and part face, used as their image names
@@ -122,7 +124,7 @@ def gen_onet_sample_data(data_dir,anno_file,det_boxs_file):
     image_done = 0
     for im_idx, dets, gts in zip(im_idx_list, det_boxes, gt_boxes_list):
         if image_done % 100 == 0:
-            print "%d images done" % image_done
+            print("%d images done" % image_done)
         image_done += 1
 
         if dets.shape[0] == 0:
@@ -194,17 +196,17 @@ def parse_args():
     parser = argparse.ArgumentParser(description='Test mtcnn',
                                      formatter_class=argparse.ArgumentDefaultsHelpFormatter)
 
-    parser.add_argument('--dataset_path', dest='dataset_path', help='dataset folder',
+    parser.add_argument('--dface_traindata_store', dest='traindata_store', help='dface train data temporary folder,include 12,24,48/postive,negative,part,landmark',
                         default='../data/wider/', type=str)
-    parser.add_argument('--anno_file', dest='annotation_file', help='output data folder',
-                        default='../data/wider/anno.txt', type=str)
+    parser.add_argument('--anno_file', dest='annotation_file', help='wider face original annotation file',
+                        default=os.path.join(config.ANNO_STORE_DIR,"wider_origin_anno.txt"), type=str)
     parser.add_argument('--pmodel_file', dest='pnet_model_file', help='PNet model file path',
-                        default='/idata/workspace/mtcnn/model_store/pnet_epoch_5best.pt', type=str)
+                        default='/idata/workspace/dface/model_store/pnet_epoch.pt', type=str)
     parser.add_argument('--rmodel_file', dest='rnet_model_file', help='RNet model file path',
-                        default='/idata/workspace/mtcnn/model_store/rnet_epoch_1.pt', type=str)
+                        default='/idata/workspace/dface/model_store/rnet_epoch.pt', type=str)
     parser.add_argument('--gpu', dest='use_cuda', help='with gpu',
                         default=config.USE_CUDA, type=bool)
-    parser.add_argument('--prefix_path', dest='prefix_path', help='image prefix root path',
+    parser.add_argument('--prefix_path', dest='prefix_path', help='annotation file image prefix root path',
                         default='', type=str)
 
     args = parser.parse_args()
@@ -214,7 +216,7 @@ def parse_args():
 
 if __name__ == '__main__':
     args = parse_args()
-    gen_onet_data(args.dataset_path, args.annotation_file, args.pnet_model_file, args.rnet_model_file, args.prefix_path, args.use_cuda)
+    gen_onet_data(args.traindata_store, args.annotation_file, args.pnet_model_file, args.rnet_model_file, args.prefix_path, args.use_cuda)
 
 
 

dface/prepare_data/gen_Pnet_train_data.py

@@ -1,14 +1,14 @@
+from __future__ import print_function
 import argparse
 import numpy as np
 import cv2
 import os
 import numpy.random as npr
-from core.utils import IoU
-import config
+from dface.core.utils import IoU
+import dface.config as config
 
 
-
-def gen_pnet_data(data_dir,anno_file):
+def gen_pnet_data(data_dir,anno_file,prefix):
 
     neg_save_dir =  os.path.join(data_dir,"12/negative")
     pos_save_dir =  os.path.join(data_dir,"12/positive")
@@ -34,7 +34,7 @@ def gen_pnet_data(data_dir,anno_file):
         annotations = f.readlines()
 
     num = len(annotations)
-    print "%d pics in total" % num
+    print("%d pics in total" % num)
     p_idx = 0
     n_idx = 0
     d_idx = 0
@@ -42,13 +42,13 @@ def gen_pnet_data(data_dir,anno_file):
     box_idx = 0
     for annotation in annotations:
         annotation = annotation.strip().split(' ')
-        im_path = annotation[0]
-        bbox = map(float, annotation[1:])
+        im_path = os.path.join(prefix,annotation[0])
+        bbox = list(map(float, annotation[1:]))
         boxes = np.array(bbox, dtype=np.int32).reshape(-1, 4)
         img = cv2.imread(im_path)
         idx += 1
         if idx % 100 == 0:
-            print idx, "images done"
+            print(idx, "images done")
 
         height, width, channel = img.shape
 
@@ -147,7 +147,7 @@ def gen_pnet_data(data_dir,anno_file):
                     cv2.imwrite(save_file, resized_im)
                     d_idx += 1
             box_idx += 1
-            print "%s images done, pos: %s part: %s neg: %s"%(idx, p_idx, d_idx, n_idx)
+            print("%s images done, pos: %s part: %s neg: %s"%(idx, p_idx, d_idx, n_idx))
 
     f1.close()
     f2.close()
@@ -159,16 +159,19 @@ def parse_args():
     parser = argparse.ArgumentParser(description='Test mtcnn',
                                      formatter_class=argparse.ArgumentDefaultsHelpFormatter)
 
-    parser.add_argument('--dataset_path', dest='dataset_path', help='dataset folder',
+    parser.add_argument('--dface_traindata_store', dest='traindata_store', help='dface train data temporary folder,include 12,24,48/postive,negative,part,landmark',
                         default='../data/wider/', type=str)
-    parser.add_argument('--anno_file', dest='annotation_file', help='dataset original annotation file',
-                        default='../data/wider/anno.txt', type=str)
-    parser.add_argument('--prefix_path', dest='prefix_path', help='image prefix root path',
+    parser.add_argument('--anno_file', dest='annotation_file', help='wider face original annotation file',
+                        default=os.path.join(config.ANNO_STORE_DIR,"wider_origin_anno.txt"), type=str)
+    parser.add_argument('--prefix_path', dest='prefix_path', help='annotation file image prefix root path',
                         default='', type=str)
 
+
+
+
     args = parser.parse_args()
     return args
 
 if __name__ == '__main__':
     args = parse_args()
-    gen_pnet_data(args.dataset_path,args.annotation_file)
+    gen_pnet_data(args.traindata_store,args.annotation_file,args.prefix_path)

dface/prepare_data/gen_Rnet_train_data.py

@@ -1,17 +1,17 @@
+from __future__ import print_function
 import argparse
-
-
 import cv2
 import numpy as np
-from core.detect import MtcnnDetector,create_mtcnn_net
-from core.imagedb import ImageDB
-from core.image_reader import TestImageLoader
+from dface.core.detect import MtcnnDetector,create_mtcnn_net
+from dface.core.imagedb import ImageDB
+from dface.core.image_reader import TestImageLoader
 import time
 import os
 import cPickle
-from core.utils import convert_to_square,IoU
-import config
-import core.vision as vision
+from dface.core.utils import convert_to_square,IoU
+import dface.config as config
+import dface.core.vision as vision
+
 
 def gen_rnet_data(data_dir, anno_file, pnet_model_file, prefix_path='', use_cuda=True, vis=False):
 
@@ -28,7 +28,7 @@ def gen_rnet_data(data_dir, anno_file, pnet_model_file, prefix_path='', use_cuda
 
     for databatch in image_reader:
         if batch_idx % 100 == 0:
-            print "%d images done" % batch_idx
+            print ("%d images done" % batch_idx)
         im = databatch
 
         t = time.time()
@@ -58,11 +58,11 @@ def gen_rnet_data(data_dir, anno_file, pnet_model_file, prefix_path='', use_cuda
         cPickle.dump(all_boxes, f, cPickle.HIGHEST_PROTOCOL)
 
 
-    gen_rnet_sample_data(data_dir,anno_file,save_file)
+    gen_rnet_sample_data(data_dir,anno_file,save_file,prefix_path)
 
 
 
-def gen_rnet_sample_data(data_dir,anno_file,det_boxs_file):
+def gen_rnet_sample_data(data_dir,anno_file,det_boxs_file,prefix_path):
 
     neg_save_dir = os.path.join(data_dir, "24/negative")
     pos_save_dir = os.path.join(data_dir, "24/positive")
@@ -85,11 +85,11 @@ def gen_rnet_sample_data(data_dir,anno_file,det_boxs_file):
     im_idx_list = list()
     gt_boxes_list = list()
     num_of_images = len(annotations)
-    print "processing %d images in total" % num_of_images
+    print ("processing %d images in total" % num_of_images)
 
     for annotation in annotations:
         annotation = annotation.strip().split(' ')
-        im_idx = annotation[0]
+        im_idx = os.path.join(prefix_path,annotation[0])
 
         boxes = map(float, annotation[1:])
         boxes = np.array(boxes, dtype=np.float32).reshape(-1, 4)
@@ -108,7 +108,7 @@ def gen_rnet_sample_data(data_dir,anno_file,det_boxs_file):
     det_handle = open(det_boxs_file, 'r')
 
     det_boxes = cPickle.load(det_handle)
-    print len(det_boxes), num_of_images
+    print(len(det_boxes), num_of_images)
     assert len(det_boxes) == num_of_images, "incorrect detections or ground truths"
 
     # index of neg, pos and part face, used as their image names
@@ -118,7 +118,7 @@ def gen_rnet_sample_data(data_dir,anno_file,det_boxs_file):
     image_done = 0
     for im_idx, dets, gts in zip(im_idx_list, det_boxes, gt_boxes_list):
         if image_done % 100 == 0:
-            print "%d images done" % image_done
+            print("%d images done" % image_done)
         image_done += 1
 
         if dets.shape[0] == 0:
@@ -194,15 +194,15 @@ def parse_args():
     parser = argparse.ArgumentParser(description='Test mtcnn',
                                      formatter_class=argparse.ArgumentDefaultsHelpFormatter)
 
-    parser.add_argument('--dataset_path', dest='dataset_path', help='dataset folder',
+    parser.add_argument('--dface_traindata_store', dest='traindata_store', help='dface train data temporary folder,include 12,24,48/postive,negative,part,landmark',
                         default='../data/wider/', type=str)
-    parser.add_argument('--anno_file', dest='annotation_file', help='dataset original annotation file',
-                        default='../data/wider/anno.txt', type=str)
+    parser.add_argument('--anno_file', dest='annotation_file', help='wider face original annotation file',
+                        default=os.path.join(config.ANNO_STORE_DIR,"wider_origin_anno.txt"), type=str)
     parser.add_argument('--pmodel_file', dest='pnet_model_file', help='PNet model file path',
-                        default='/idata/workspace/mtcnn/model_store/pnet_epoch_5best.pt', type=str)
+                        default='/idata/workspace/dface/model_store/pnet_epoch.pt', type=str)
     parser.add_argument('--gpu', dest='use_cuda', help='with gpu',
                         default=config.USE_CUDA, type=bool)
-    parser.add_argument('--prefix_path', dest='prefix_path', help='image prefix root path',
+    parser.add_argument('--prefix_path', dest='prefix_path', help='annotation file image prefix root path',
                         default='', type=str)
 
 
@@ -213,7 +213,7 @@ def parse_args():
 
 if __name__ == '__main__':
     args = parse_args()
-    gen_rnet_data(args.dataset_path, args.annotation_file, args.pnet_model_file, args.prefix_path, args.use_cuda)
+    gen_rnet_data(args.traindata_store, args.annotation_file, args.pnet_model_file, args.prefix_path, args.use_cuda)
 
 
 

dface/prepare_data/gen_landmark_12.py

@@ -1,12 +1,13 @@
 # coding: utf-8
+from __future__ import print_function
 import os
 import cv2
 import numpy as np
 import sys
 import numpy.random as npr
 import argparse
-import config
-import core.utils as utils
+import dface.config as config
+import dface.core.utils as utils
 
 
 def gen_data(anno_file, data_dir, prefix):
@@ -34,7 +35,7 @@ def gen_data(anno_file, data_dir, prefix):
         annotations = f2.readlines()
 
     num = len(annotations)
-    print "%d pics in total" % num
+    print("%d pics in total" % num)
 
     l_idx =0
     idx = 0
@@ -68,7 +69,7 @@ def gen_data(anno_file, data_dir, prefix):
 
         idx = idx + 1
         if idx % 100 == 0:
-            print "%d images done, landmark images: %d"%(idx,l_idx)
+            print("%d images done, landmark images: %d"%(idx,l_idx))
 
         x1, y1, x2, y2 = gt_box
 
@@ -137,11 +138,11 @@ def parse_args():
     parser = argparse.ArgumentParser(description='Test mtcnn',
                                      formatter_class=argparse.ArgumentDefaultsHelpFormatter)
 
-    parser.add_argument('--dataset_path', dest='dataset_path', help='dataset folder',
+    parser.add_argument('--dface_traindata_store', dest='traindata_store', help='dface train data temporary folder,include 12,24,48/postive,negative,part,landmark',
                         default='../data/wider/', type=str)
-    parser.add_argument('--anno_file', dest='annotation_file', help='dataset original annotation file',
+    parser.add_argument('--anno_file', dest='annotation_file', help='celeba dataset original annotation file',
                         default='../data/wider/anno.txt', type=str)
-    parser.add_argument('--prefix_path', dest='prefix_path', help='image prefix root path',
+    parser.add_argument('--prefix_path', dest='prefix_path', help='annotation file image prefix root path',
                         default='../data/', type=str)
 
 
@@ -151,6 +152,6 @@ def parse_args():
 if __name__ == '__main__':
     args = parse_args()
 
-    gen_data(args.annotation_file, args.dataset_path, args.prefix_path)
+    gen_data(args.annotation_file, args.traindata_store, args.prefix_path)
 
 

dface/prepare_data/gen_landmark_24.py

@@ -1,4 +1,5 @@
 # coding: utf-8
+from __future__ import print_function
 import os
 import cv2
 import numpy as np
@@ -6,8 +7,8 @@ import random
 import sys
 import numpy.random as npr
 import argparse
-import config
-import core.utils as utils
+import dface.config as config
+import dface.core.utils as utils
 
 
 
@@ -35,7 +36,7 @@ def gen_data(anno_file, data_dir, prefix):
         annotations = f2.readlines()
 
     num = len(annotations)
-    print "%d total images" % num
+    print("%d total images" % num)
 
     l_idx =0
     idx = 0
@@ -67,7 +68,7 @@ def gen_data(anno_file, data_dir, prefix):
 
         idx = idx + 1
         if idx % 100 == 0:
-            print "%d images done, landmark images: %d"%(idx,l_idx)
+            print("%d images done, landmark images: %d"%(idx,l_idx))
 
         x1, y1, x2, y2 = gt_box
 
@@ -135,11 +136,11 @@ def parse_args():
     parser = argparse.ArgumentParser(description='Test mtcnn',
                                      formatter_class=argparse.ArgumentDefaultsHelpFormatter)
 
-    parser.add_argument('--dataset_path', dest='dataset_path', help='dataset folder',
+    parser.add_argument('--dface_traindata_store', dest='traindata_store', help='dface train data temporary folder,include 12,24,48/postive,negative,part,landmark',
                         default='/idata/data/wider/', type=str)
-    parser.add_argument('--anno_file', dest='annotation_file', help='dataset original annotation file',
+    parser.add_argument('--anno_file', dest='annotation_file', help='celeba dataset original annotation file',
                         default='/idata/data/trainImageList.txt', type=str)
-    parser.add_argument('--prefix_path', dest='prefix_path', help='image prefix root path',
+    parser.add_argument('--prefix_path', dest='prefix_path', help='annotation file image prefix root path',
                         default='/idata/data', type=str)
 
 
@@ -149,6 +150,6 @@ def parse_args():
 if __name__ == '__main__':
     args = parse_args()
 
-    gen_data(args.annotation_file, args.dataset_path, args.prefix_path)
+    gen_data(args.annotation_file, args.traindata_store, args.prefix_path)