tensorflow - SimpleTransformers Error: VersionConflict: tokenizers==0.9.4? How do I fix this?

Question

I'm trying to execute the simpletransformers example from their site on google colab.

Example:

from simpletransformers.classification import ClassificationModel, ClassificationArgs
import pandas as pd
import logging


logging.basicConfig(level=logging.INFO)
transformers_logger = logging.getLogger("transformers")
transformers_logger.setLevel(logging.WARNING)

# Preparing train data
train_data = [
    ["Aragorn was the heir of Isildur", 1],
    ["Frodo was the heir of Isildur", 0],
]
train_df = pd.DataFrame(train_data)
train_df.columns = ["text", "labels"]

# Preparing eval data
eval_data = [
    ["Theoden was the king of Rohan", 1],
    ["Merry was the king of Rohan", 0],
]
eval_df = pd.DataFrame(eval_data)
eval_df.columns = ["text", "labels"]

# Optional model configuration
model_args = ClassificationArgs(num_train_epochs=1)

# Create a ClassificationModel
model = ClassificationModel(
    "roberta", "roberta-base", args=model_args
)

# Train the model
model.train_model(train_df)

# Evaluate the model
result, model_outputs, wrong_predictions = model.eval_model(eval_df)

# Make predictions with the model
predictions, raw_outputs = model.predict(["Sam was a Wizard"])

But it gives me the following error:

VersionConflict: tokenizers==0.9.4 is required for a normal functioning of this module, but found tokenizers==0.10.0. Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git master

I've tried !pip install transformers -Uand even !pip install tokenizers==0.9.4 but keeps giving the same error. I have executed this code before and it worked just fun, but now it's giving the mentioned error.

question from:https://stackoverflow.com/questions/65924090/simpletransformers-error-versionconflict-tokenizers-0-9-4-how-do-i-fix-this

Answer 1

I am putting this here incase someone faces the same problem. I was helped by the creator himself.

Workaround:
Install tokenizers==0.9.4 before install simpletransformers

In Colab for example;

!pip install tokenizers==0.9.4
!pip install simpletransformers

https://github.com/ThilinaRajapakse/simpletransformers/issues/950