NameError: 名称 'tokenize' 未定义
问题描述:
到情感分析分类:我们训练了各种机器学习模型和深度神经网络,将评论文本的情感分为 3 个主要类别(正面、中性、负面).
To to Sentiment Analysis Classification : We trained various machine learning models and deep neural networks to classify the sentiment of review text into 3 main categories (Positive, Neutral, Negative).
# TfidfVectorizer transform train and test
con_vec = TfidfVectorizer(stop_words='english',tokenizer=tokenize,max_features=20000,ngram_range=(1,2))
X_train_tfidf = con_vec.fit_transform(X_train)
# with open("tfidf_vectorizer.pkl", 'wb') as handle:
# pickle.dump(con_vec, handle)
y_train_tfidf = y_train
X_test_tfidf = con_vec.transform(X_test)
y_test_tfidf = y_test
**ERROR CODE :**
**Pandas Version is pandas 1.1.4**
---------------------------------------------------------------------------
1 from nltk import word_tokenize # Generate table of words with their counts
2 from nltk.tokenize import word_tokenize # TfidfVectorizer transform train and test
----> 3 con_vec = TfidfVectorizer(stop_words='english',tokenizer=tokenize,max_features=20000,ngram_range=(1,2))
4 X_train_tfidf = con_vec.fit_transform(X_train)
**NameError: name 'tokenize' is not defined**
Any solution how to remove this error?
Also tried from **nltk import word_tokenize** but still same error
[1]: https://i.stack.imgur.com/lmoSp.png
答
尝试导入:
from nltk.tokenize import tokenize