Doc2vec: Wie man Dokumentenvektoren erhält

Wenn Sie trainieren möchten, Doc2Vec Modell, Datensatz muss enthalten Listen von Wörtern (ähnlich wie Word2Vec-format) und tags (id-Dokumente). Es kann auch enthalten Sie einige zusätzliche Informationen (siehe https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/doc2vec-IMDB.ipynb für weitere Informationen).

# Import libraries

from gensim.models import doc2vec
from collections import namedtuple

# Load data

doc1 = ["This is a sentence", "This is another sentence"]

# Transform data (you can add more data preprocessing steps) 

docs = []
analyzedDocument = namedtuple('AnalyzedDocument', 'words tags')
for i, text in enumerate(doc1):
    words = text.lower().split()
    tags = [i]
    docs.append(analyzedDocument(words, tags))

# Train model (set min_count = 1, if you want the model to work with the provided example data set)

model = doc2vec.Doc2Vec(docs, size = 100, window = 300, min_count = 1, workers = 4)

# Get the vectors

model.docvecs[0]
model.docvecs[1]

UPDATE (how to train in Epochen):
Doc2Vec Funktion enthält alpha und min_alpha Parameter, aber das bedeutet, dass die learning-rate zerfällt während einer Epoche von alpha zu min_alpha. Trainieren mehrere Epochen, legen Sie die learning-rate manuell, so wie hier:

from gensim.models import doc2vec
import random

alpha_val = 0.025        # Initial learning rate
min_alpha_val = 1e-4     # Minimum for linear learning rate decay
passes = 15              # Number of passes of one document during training

alpha_delta = (alpha_val - min_alpha_val) / (passes - 1)

model = doc2vec.Doc2Vec( size = 100 # Model initialization
    , window = 300
    , min_count = 1
    , workers = 4)

model.build_vocab(docs) # Building vocabulary

for epoch in range(passes):

    # Shuffling gets better results

    random.shuffle(docs)

    # Train

    model.alpha, model.min_alpha = alpha_val, alpha_val

    model.train(docs)

    # Logs

    print('Completed pass %i at alpha %f' % (epoch + 1, alpha_val))

    # Next run alpha

    alpha_val -= alpha_delta

InformationsquelleAutor der Antwort Lenka Vraná