Skip to content

Commit c706141

Browse files
committed
BUG Output is in bz2 format
1 parent d264e08 commit c706141

3 files changed

Lines changed: 6 additions & 4 deletions

File tree

‎ch04/README.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,5 +18,7 @@ Run the following two commands inside the ``data/`` directory::
1818

1919
As the filenames indicate, the first step will download the data and the second
2020
one will preprocess it. Preprocessing can take several hours, but it is
21-
feasible to run it on a modern laptop.
21+
feasible to run it on a modern laptop. Once the second step is finished, you
22+
may remove the input file if you want to save disk space
23+
(``data/enwiki-latest-pages-articles.xml.bz2``).
2224

‎ch04/data/.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,4 @@ wiki_en_output_bow.mm
55
wiki_en_output_bow.mm.index
66
wiki_en_output_tfidf.mm
77
wiki_en_output_tfidf.mm.index
8-
wiki_en_output_wordids.txt
8+
wiki_en_output_wordids.txt.bz2

‎ch04/wikitopics.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
format='%(asctime)s : %(levelname)s : %(message)s',
1414
level=logging.INFO)
1515
id2word = gensim.corpora.Dictionary.load_from_text(
16-
'data/wiki_en_output_wordids.txt')
16+
'data/wiki_en_output_wordids.txt.bz2')
1717
mm = gensim.corpora.MmCorpus('data/wiki_en_output_tfidf.mm')
1818
model = gensim.models.ldamodel.LdaModel(
1919
corpus=mm,
@@ -30,7 +30,7 @@
3030

3131
counts = np.zeros(100)
3232
for doc_top in topics:
33-
for ti, _ in doc_toc:
33+
for ti, _ in doc_top:
3434
counts[ti] += 1
3535

3636
for doc_top in topics:

0 commit comments

Comments
 (0)