python - TypeError: file must have a 'write' attribute

Question

I am trying to train a FastText model.

from gensim.test.utils import datapath
from gensim.models.fasttext import FastText
from gensim.test.utils import get_tmpfile
embedding_size = 300
window_size = 10
min_word = 5
down_sampling = 1e-2
corpus_file = datapath('data.cor')
model = FastText(size=embedding_size,
                  window=window_size,
                  min_count=min_word,
                  sample=down_sampling,
                  sg=1,
                  iter=100)
model.build_vocab(corpus_file=corpus_file)
total_words = model.corpus_total_words
model.train(corpus_file=corpus_file, total_words=total_words, epochs=5)
fname = get_tmpfile("fasttext.model")
model.save(fname)

However, the following error appears:

Traceback (most recent call last):
  File "C:ProgramDataAnaconda3libsite-packagesgensimutils.py", line 692, in save
    _pickle.dump(self, fname_or_handle, protocol=pickle_protocol)
TypeError: file must have a 'write' attribute

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:/untitled1/dfgd.py", line 21, in <module>
    model.save(fname)
  File "C:ProgramDataAnaconda3libsite-packagesgensimmodelsfasttext.py", line 1017, in save
    super(FastText, self).save(*args, **kwargs)
  File "C:ProgramDataAnaconda3libsite-packagesgensimmodelsase_any2vec.py", line 621, in save
    super(BaseAny2VecModel, self).save(fname_or_handle, **kwargs)
  File "C:ProgramDataAnaconda3libsite-packagesgensimutils.py", line 695, in save
    self._smart_save(fname_or_handle, separately, sep_limit, ignore, pickle_protocol=pickle_protocol)
  File "C:ProgramDataAnaconda3libsite-packagesgensimutils.py", line 547, in _smart_save
compress, subname)
  File "C:ProgramDataAnaconda3libsite-packagesgensimutils.py", line 609, in _save_specials
restores.extend(val._save_specials(cfname, None, sep_limit, ignore, pickle_protocol, compress, subname))
  File "C:ProgramDataAnaconda3libsite-packagesgensimutils.py", line 621, in _save_specials
    np.save(subname(fname, attrib), np.ascontiguousarray(val))
  File "C:ProgramDataAnaconda3libsite-packages
umpylib
pyio.py", line 536, in save
    pickle_kwargs=pickle_kwargs)
  File "C:ProgramDataAnaconda3libsite-packages
umpylibformat.py", line 644, in write_array
    array.tofile(fp)
OSError: 418859700 requested and 0 written

I have previously trained the model with the window size set to be 9 and less. There was no problem at that time and no error appeared. How can I fix this error?

question from:https://stackoverflow.com/questions/65924072/typeerror-file-must-have-a-write-attribute

Answer 1

Why do you want to write your large model file to a "temporary" directory? Wouldn't you prefer to save it to an explicit place under your control? The .save() method will take any path of your choosing.

Also note that anytime you get such an error, it may be hiding some other disk-IO problem, like insufficient privileges for the current user, insufficient space for the data, an illegal path type for the named volume, an existing unchangeable file, etc.

Thus is can be helpful to:

• check what actual exact path was involved (the value of fname)

• try other more-simple operations on that same path, either from Python code, or outside of it. (Can you use basic Python operations to write a single line to that path? Can you use shell operations to list that path's enclosing directory, or create a stub file at that location? etc.) You may get a more clear error from those other attempts, which shows the real problem.

Finally, though things like Gensim, Python, & the various scientific-computing libraries they depend upon are all officially supported on Windows, they tend to get far more developer attention, real-world-use, & testing on Linux/Unix-based systems – so lots of the example/tutorial code, and suggestions from quality answers, will work best there. If now or in the future you have the choice to use a non-Windows system for this kind of work, lots of little problems may be avoided.