Summary:
This fixes a race condition in text_file_reader.py.
For example in `fbcode/caffe2/caffe2/fb/text/stats.py`, in `compute_meta`, we build an execution step `read` such as:
```
.
└── step_read
├── net_reader
│ ├── op_TextFileReaderRead
│ └── op_IsEmpty
└── net_consume:n
└── op_Tokenize
```
Note that in `workspace.cc`, we check should_stop between each _step_ and each _net_, not between _ops_
Let's say we have 2 workers, here is a faulty interleaving of threads:
- 1 executes TextFileReaderRead
- 2 executes TextFileReaderRead
- 1 executes IsEmpty and sets should_stop to False
- 2 executes IsEmpty and sets should_stop to True
- 1 checks should_stop before running net_consume:n
- 1 stops
- 2 checks should_stop before running net_consume:n
- 2 stops
That's an issue, because 1 did read data from the file but did not run the processing step (consume:n) for this data.
Reviewed By: dzhulgakov
Differential Revision: D4203729
fbshipit-source-id: eabd94ea995527ec52fa137a8b63c277f7e4dd96
