Commits

Commits on Dec 20, 2016

added print statement to print the bigram perplexity on the actual data set file 'test.txt'. This completes the project :)
Oliver Holland
committed
391cc8b
added print statement to print the unigram perplexity on the actual data set file 'test.txt'
Oliver Holland
committed
024716c
added print statement to print the bigram perplexity on the actual data set file 'train.txt'
Oliver Holland
committed
00646d1
added print statement to print the unigram perplexity on the actual data set file 'train.txt'
Oliver Holland
committed
394cfa9
added function to calculate bigram perplexity on test data along with appropriate print call
Oliver Holland
committed
dd90e6c
moved calulate number of unigrams and bigrams out of their resepective classes. Created a new function that calulates the perplexity for unigram. Added necessary print statement in main.
Oliver Holland
committed
54bb623
created a function to calculate bigram sentence probabilities and added the appropriate methods to print so it formats nicely.
Oliver Holland
committed
ad4a117
in main, added the necessary functionality to print the unigram sentence proababilities. calculations can be confirmed to be correct
Oliver Holland
committed
1621c2f
added a definition to calculate the sentence probability for unigrams
Oliver Holland
committed
725be19

Commits on Nov 29, 2016

added a print bigram probabilities function and called it in main. Formatted print so it displays in the said format
Oliver Holland
committed
dc794d0
created a new def to calculate the number of bigrams based on the sentences and returned its count. Created another def to calculate the bigram probability.
Oliver Holland
committed
09e2b5d
Created a new class for the Bigram Language Model. Similar to the the unigram model class, however need to take into account the previous word.
Oliver Holland
committed
cb97edd
added smoothed probabilites in main
Oliver Holland
committed
cb8dd20
Created a definiton that sorts the vocab file and appends to list with associated keys. Created a def that prints the unigram probs checking tomake sure the start and end sentence symbols are not t…
Oliver Holland
committed
21e16f0
Created a def to calculate unigram probability. Changed hard coded file path to be called in function later in main.
Oliver Holland
committed
9623a33
created UnigramLanguageModel class. Created a def to calculate number of unigrams (number of tokens in corpus)
Oliver Holland
committed
edf33b8
created a defintion that reads sentences from sampledata.txt and splits them by new line
Oliver Holland
committed
fb34487
added data sets (txt files) and created a python file
Oliver Holland
committed
1936397
Update README.md
Oliver Holland
committed
8c2c527
Update README.md
Oliver Holland
committed
378cf43
Update README.md for better presentation and structure
Oliver Holland
committed
0e48ec0
Add new file
Oliver Holland
committed
0d4b2c1