russian_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/russian/stop.txt
и              | and
в              | in/into
во             | alternative form
не             | not
что            | what/that
он             | he
на             | on/onto
я              | i
с              | from
со             | alternative form
как            | how
а              | milder form of `no' (but)
то             | conjunction and form of `that'
все            | all
она            | she
так            | so, thus
его            | him
но             | but
да             | yes/and
ты             | thou
к              | towards, by
у              | around, chez
же             | intensifier particle
вы             | you
за             | beyond, behind
бы             | conditional/subj. particle
по             | up to, along
только         | only
ее             | her
мне            | to me
было           | it was
вот            | here is/are, particle
от             | away from
меня           | me
еще            | still, yet, more
нет            | no, there isnt/arent
о              | about
из             | out of
ему            | to him
теперь         | now
когда          | when
даже           | even
ну             | so, well
вдруг          | suddenly
ли             | interrogative particle
если           | if
уже            | already, but homonym of `narrower'
или            | or
ни             | neither
быть           | to be
был            | he was
него           | prepositional form of его
до             | up to
вас            | you accusative
нибудь         | indef. suffix preceded by hyphen
опять          | again
уж             | already, but homonym of `adder'
вам            | to you
сказал         | he said
ведь           | particle `after all'
там            | there
потом          | then
себя           | oneself
ничего         | nothing
ей             | to her
может          | usually with `быть' as `maybe'
они            | they
тут            | here
где            | where
есть           | there is/are
надо           | got to, must
ней            | prepositional form of  ей
для            | for
мы             | we
тебя           | thee
их             | them, their
чем            | than
была           | she was
сам            | self
чтоб           | in order to
без            | without
будто          | as if
человек        | man, person, one
чего           | genitive form of `what'
раз            | once
тоже           | also
себе           | to oneself
под            | beneath
жизнь          | life
будет          | will be
ж              | short form of intensifer particle `же'
тогда          | then
кто            | who
этот           | this
говорил        | was saying
того           | genitive form of `that'
потому         | for that reason
этого          | genitive form of `this'
#.........这里部分代码省略.........

danish_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/danish/stop.txt
og           | and
i            | in
jeg          | I
det          | that (dem. pronoun)/it (pers. pronoun)
at           | that (in front of a sentence)/to (with infinitive)
en           | a/an
den          | it (pers. pronoun)/that (dem. pronoun)
til          | to/at/for/until/against/by/of/into, more
er           | present tense of "to be"
som          | who, as
på           | on/upon/in/on/at/to/after/of/with/for, on
de           | they
med          | with/by/in, along
han          | he
af           | of/by/from/off/for/in/with/on, off
for          | at/for/to/from/by/of/ago, in front/before, because
ikke         | not
der          | who/which, there/those
var          | past tense of "to be"
mig          | me/myself
sig          | oneself/himself/herself/itself/themselves
men          | but
et           | a/an/one, one (number), someone/somebody/one
har          | present tense of "to have"
om           | round/about/for/in/a, about/around/down, if
vi           | we
min          | my
havde        | past tense of "to have"
ham          | him
hun          | she
nu           | now
over         | over/above/across/by/beyond/past/on/about, over/past
da           | then, when/as/since
fra          | from/off/since, off, since
du           | you
ud           | out
sin          | his/her/its/one's
dem          | them
os           | us/ourselves
op           | up
man          | you/one
hans         | his
hvor         | where
eller        | or
hvad         | what
skal         | must/shall etc.
selv         | myself/youself/herself/ourselves etc., even
her          | here
alle         | all/everyone/everybody etc.
vil          | will (verb)
blev         | past tense of "to stay/to remain/to get/to become"
kunne        | could
ind          | in
når          | when
være         | present tense of "to be"
dog          | however/yet/after all
noget        | something
ville        | would
jo           | you know/you see (adv), yes
deres        | their/theirs
efter        | after/behind/according to/for/by/from, later/afterwards
ned          | down
skulle       | should
denne        | this
end          | than
dette        | this
mit          | my/mine
også         | also
under        | under/beneath/below/during, below/underneath
have         | have
dig          | you
anden        | other
hende        | her
mine         | my
alt          | everything
meget        | much/very, plenty of
sit          | his, her, its, one's
sine         | his, her, its, one's
vor          | our
mod          | against
disse        | these
hvis         | if
din          | your/yours
nogle        | some
hos          | by/at
blive        | be/become
mange        | many
ad           | by/through
bliver       | present tense of "to be/to become"
hendes       | her/hers
været        | be
thi          | for (conj)
jer          | you
sådan        | such, like this/like that
''')

italian_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/italian/stop.txt
ad             |  a (to) before vowel
al             |  a + il
allo           |  a + lo
ai             |  a + i
agli           |  a + gli
all            |  a + l'
agl            |  a + gl'
alla           |  a + la
alle           |  a + le
con            |  with
col            |  con + il
coi            |  con + i (forms collo, cogli etc are now very rare)
da             |  from
dal            |  da + il
dallo          |  da + lo
dai            |  da + i
dagli          |  da + gli
dall           |  da + l'
dagl           |  da + gll'
dalla          |  da + la
dalle          |  da + le
di             |  of
del            |  di + il
dello          |  di + lo
dei            |  di + i
degli          |  di + gli
dell           |  di + l'
degl           |  di + gl'
della          |  di + la
delle          |  di + le
in             |  in
nel            |  in + el
nello          |  in + lo
nei            |  in + i
negli          |  in + gli
nell           |  in + l'
negl           |  in + gl'
nella          |  in + la
nelle          |  in + le
su             |  on
sul            |  su + il
sullo          |  su + lo
sui            |  su + i
sugli          |  su + gli
sull           |  su + l'
sugl           |  su + gl'
sulla          |  su + la
sulle          |  su + le
per            |  through, by
tra            |  among
contro         |  against
io             |  I
tu             |  thou
lui            |  he
lei            |  she
noi            |  we
voi            |  you
loro           |  they
mio            |  my
mia            |
miei           |
mie            |
tuo            |
tua            |
tuoi           |  thy
tue            |
suo            |
sua            |
suoi           |  his, her
sue            |
nostro         |  our
nostra         |
nostri         |
nostre         |
vostro         |  your
vostra         |
vostri         |
vostre         |
mi             |  me
ti             |  thee
ci             |  us, there
vi             |  you, there
lo             |  him, the
la             |  her, the
li             |  them
le             |  them, the
gli            |  to him, the
ne             |  from there etc
il             |  the
un             |  a
uno            |  a
una            |  a
ma             |  but
ed             |  and
se             |  if
perché         |  why, because
anche          |  also
come           |  how
#.........这里部分代码省略.........

spanish_stopwords = parse_stop_word(u'''
|source: http://snowball.tartarus.org/algorithms/spanish/stop.txt
de             |  from, of
la             |  the, her
que            |  who, that
el             |  the
en             |  in
y              |  and
a              |  to
los            |  the, them
del            |  de + el
se             |  himself, from him etc
las            |  the, them
por            |  for, by, etc
un             |  a
para           |  for
con            |  with
no             |  no
una            |  a
su             |  his, her
al             |  a + el
  | es         from SER
lo             |  him
como           |  how
más            |  more
pero           |  pero
sus            |  su plural
le             |  to him, her
ya             |  already
o              |  or
  | fue        from SER
este           |  this
  | ha         from HABER
sí             |  himself etc
porque         |  because
esta           |  this
  | son        from SER
entre          |  between
  | está     from ESTAR
cuando         |  when
muy            |  very
sin            |  without
sobre          |  on
  | ser        from SER
  | tiene      from TENER
también        |  also
me             |  me
hasta          |  until
hay            |  there is/are
donde          |  where
  | han        from HABER
quien          |  whom, that
  | están      from ESTAR
  | estado     from ESTAR
desde          |  from
todo           |  all
nos            |  us
durante        |  during
  | estados    from ESTAR
todos          |  all
uno            |  a
les            |  to them
ni             |  nor
contra         |  against
otros          |  other
  | fueron     from SER
ese            |  that
eso            |  that
  | había      from HABER
ante           |  before
ellos          |  they
e              |  and (variant of y)
esto           |  this
mí             |  me
antes          |  before
algunos        |  some
qué            |  what?
unos           |  a
yo             |  I
otro           |  other
otras          |  other
otra           |  other
él             |  he
tanto          |  so much, many
esa            |  that
estos          |  these
mucho          |  much, many
quienes        |  who
nada           |  nothing
muchos         |  many
cual           |  who
  | sea        from SER
poco           |  few
ella           |  she
estar          |  to be
  | haber      from HABER
estas          |  these
  | estaba     from ESTAR
  | estamos    from ESTAR
algunas        |  some
#.........这里部分代码省略.........

portuguese_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/portuguese/stop.txt
de             |  of, from
a              |  the; to, at; her
o              |  the; him
que            |  who, that
e              |  and
do             |  de + o
da             |  de + a
em             |  in
um             |  a
para           |  for
  | é          from SER
com            |  with
não            |  not, no
uma            |  a
os             |  the; them
no             |  em + o
se             |  himself etc
na             |  em + a
por            |  for
mais           |  more
as             |  the; them
dos            |  de + os
como           |  as, like
mas            |  but
  | foi        from SER
ao             |  a + o
ele            |  he
das            |  de + as
  | tem        from TER
à              |  a + a
seu            |  his
sua            |  her
ou             |  or
  | ser        from SER
quando         |  when
muito          |  much
  | há         from HAV
nos            |  em + os; us
já             |  already, now
  | está       from EST
eu             |  I
também         |  also
só             |  only, just
pelo           |  per + o
pela           |  per + a
até            |  up to
isso           |  that
ela            |  he
entre          |  between
  | era        from SER
depois         |  after
sem            |  without
mesmo          |  same
aos            |  a + os
  | ter        from TER
seus           |  his
quem           |  whom
nas            |  em + as
me             |  me
esse           |  that
eles           |  they
  | estão      from EST
você           |  you
  | tinha      from TER
  | foram      from SER
essa           |  that
num            |  em + um
nem            |  nor
suas           |  her
meu            |  my
às             |  a + as
minha          |  my
  | têm        from TER
numa           |  em + uma
pelos          |  per + os
elas           |  they
  | havia      from HAV
  | seja       from SER
qual           |  which
  | será       from SER
nós            |  we
  | tenho      from TER
lhe            |  to him, her
deles          |  of them
essas          |  those
esses          |  those
pelas          |  per + as
este           |  this
  | fosse      from SER
dele           |  of him

 | other words. There are many contractions such as naquele = em+aquele,
 | mo = me+o, but they are rare.
 | Indefinite article plural forms are also rare.

tu             |  thou
te             |  thee
vocês          |  you (plural)
#.........这里部分代码省略.........

french_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/french/stop.txt
au             |  a + le
aux            |  a + les
avec           |  with
ce             |  this
ces            |  these
dans           |  with
de             |  of
des            |  de + les
du             |  de + le
elle           |  she
en             |  `of them' etc
et             |  and
eux            |  them
il             |  he
je             |  I
la             |  the
le             |  the
leur           |  their
lui            |  him
ma             |  my (fem)
mais           |  but
me             |  me
même           |  same; as in moi-même (myself) etc
mes            |  me (pl)
moi            |  me
mon            |  my (masc)
ne             |  not
nos            |  our (pl)
notre          |  our
nous           |  we
on             |  one
ou             |  where
par            |  by
pas            |  not
pour           |  for
qu             |  que before vowel
que            |  that
qui            |  who
sa             |  his, her (fem)
se             |  oneself
ses            |  his (pl)
son            |  his, her (masc)
sur            |  on
ta             |  thy (fem)
te             |  thee
tes            |  thy (pl)
toi            |  thee
ton            |  thy (masc)
tu             |  thou
un             |  a
une            |  a
vos            |  your (pl)
votre          |  your
vous           |  you

               |  single letter forms

c              |  c'
d              |  d'
j              |  j'
l              |  l'
à              |  to, at
m              |  m'
n              |  n'
s              |  s'
t              |  t'
y              |  there

               | forms of être (not including the infinitive):
été
étée
étées
étés
étant
suis
es
est
sommes
êtes
sont
serai
seras
sera
serons
serez
seront
serais
serait
serions
seriez
seraient
étais
était
étions
étiez
étaient
fus
fut
#.........这里部分代码省略.........

hungarian_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/hungarian/stop.txt
| prepared by Anna Tordai
a
ahogy
ahol
aki
akik
akkor
alatt
által
általában
amely
amelyek
amelyekben
amelyeket
amelyet
amelynek
ami
amit
amolyan
amíg
amikor
át
abban
ahhoz
annak
arra
arról
az
azok
azon
azt
azzal
azért
aztán
azután
azonban
bár
be
belül
benne
cikk
cikkek
cikkeket
csak
de
e
eddig
egész
egy
egyes
egyetlen
egyéb
egyik
egyre
ekkor
el
elég
ellen
elő
először
előtt
első
én
éppen
ebben
ehhez
emilyen
ennek
erre
ez
ezt
ezek
ezen
ezzel
ezért
és
fel
felé
hanem
hiszen
hogy
hogyan
igen
így
illetve
ill.
ill
ilyen
ilyenkor
ison
ismét
itt
jó
jól
jobban
kell
kellett
keresztül
#.........这里部分代码省略.........

dutch_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/dutch/stop.txt
de             |  the
en             |  and
van            |  of, from
ik             |  I, the ego
te             |  (1) chez, at etc, (2) to, (3) too
dat            |  that, which
die            |  that, those, who, which
in             |  in, inside
een            |  a, an, one
hij            |  he
het            |  the, it
niet           |  not, nothing, naught
zijn           |  (1) to be, being, (2) his, one's, its
is             |  is
was            |  (1) was, past tense of all persons sing. of 'zijn' (to be) (2) wax, (3) the washing, (4) rise of river
op             |  on, upon, at, in, up, used up
aan            |  on, upon, to (as dative)
met            |  with, by
als            |  like, such as, when
voor           |  (1) before, in front of, (2) furrow
had            |  had, past tense all persons sing. of 'hebben' (have)
er             |  there
maar           |  but, only
om             |  round, about, for etc
hem            |  him
dan            |  then
zou            |  should/would, past tense all persons sing. of 'zullen'
of             |  or, whether, if
wat            |  what, something, anything
mijn           |  possessive and noun 'mine'
men            |  people, 'one'
dit            |  this
zo             |  so, thus, in this way
door           |  through by
over           |  over, across
ze             |  she, her, they, them
zich           |  oneself
bij            |  (1) a bee, (2) by, near, at
ook            |  also, too
tot            |  till, until
je             |  you
mij            |  me
uit            |  out of, from
der            |  Old Dutch form of 'van der' still found in surnames
daar           |  (1) there, (2) because
haar           |  (1) her, their, them, (2) hair
naar           |  (1) unpleasant, unwell etc, (2) towards, (3) as
heb            |  present first person sing. of 'to have'
hoe            |  how, why
heeft          |  present third person sing. of 'to have'
hebben         |  'to have' and various parts thereof
deze           |  this
u              |  you
want           |  (1) for, (2) mitten, (3) rigging
nog            |  yet, still
zal            |  'shall', first and third person sing. of verb 'zullen' (will)
me             |  me
zij            |  she, they
nu             |  now
ge             |  'thou', still used in Belgium and south Netherlands
geen           |  none
omdat          |  because
iets           |  something, somewhat
worden         |  to become, grow, get
toch           |  yet, still
al             |  all, every, each
waren          |  (1) 'were' (2) to wander, (3) wares, (3)
veel           |  much, many
meer           |  (1) more, (2) lake
doen           |  to do, to make
toen           |  then, when
moet           |  noun 'spot/mote' and present form of 'to must'
ben            |  (1) am, (2) 'are' in interrogative second person singular of 'to be'
zonder         |  without
kan            |  noun 'can' and present form of 'to be able'
hun            |  their, them
dus            |  so, consequently
alles          |  all, everything, anything
onder          |  under, beneath
ja             |  yes, of course
eens           |  once, one day
hier           |  here
wie            |  who
werd           |  imperfect third person sing. of 'become'
altijd         |  always
doch           |  yet, but etc
wordt          |  present third person sing. of 'become'
wezen          |  (1) to be, (2) 'been' as in 'been fishing', (3) orphans
kunnen         |  to be able
ons            |  us/our
zelf           |  self
tegen          |  against, towards, at
na             |  after, near
reeds          |  already
wil            |  (1) present tense of 'want', (2) 'will', noun, (3) fender
kon            |  could; past tense of 'to be able'
niets          |  nothing
uw             |  your
#.........这里部分代码省略.........

norwegian_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/norwegian/stop.txt
og             | and
i              | in
jeg            | I
det            | it/this/that
at             | to (w. inf.)
en             | a/an
et             | a/an
den            | it/this/that
til            | to
er             | is/am/are
som            | who/that
på             | on
de             | they / you(formal)
med            | with
han            | he
av             | of
ikke           | not
ikkje          | not *
der            | there
så             | so
var            | was/were
meg            | me
seg            | you
men            | but
ett            | one
har            | have
om             | about
vi             | we
min            | my
mitt           | my
ha             | have
hadde          | had
hun            | she
nå             | now
over           | over
da             | when/as
ved            | by/know
fra            | from
du             | you
ut             | out
sin            | your
dem            | them
oss            | us
opp            | up
man            | you/one
kan            | can
hans           | his
hvor           | where
eller          | or
hva            | what
skal           | shall/must
selv           | self (reflective)
sjøl           | self (reflective)
her            | here
alle           | all
vil            | will
bli            | become
ble            | became
blei           | became *
blitt          | have become
kunne          | could
inn            | in
når            | when
være           | be
kom            | come
noen           | some
noe            | some
ville          | would
dere           | you
som            | who/which/that
deres          | their/theirs
kun            | only/just
ja             | yes
etter          | after
ned            | down
skulle         | should
denne          | this
for            | for/because
deg            | you
si             | hers/his
sine           | hers/his
sitt           | hers/his
mot            | against
å              | to
meget          | much
hvorfor        | why
dette          | this
disse          | these/those
uten           | without
hvordan        | how
ingen          | none
din            | your
ditt           | your
blir           | become
samme          | same
hvilken        | which
hvilke         | which (plural)
sånn           | such a
#.........这里部分代码省略.........

swedish_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/swedish/stop.txt
och            | and
det            | it, this/that
att            | to (with infinitive)
i              | in, at
en             | a
jag            | I
hon            | she
som            | who, that
han            | he
på             | on
den            | it, this/that
med            | with
var            | where, each
sig            | him(self) etc
för            | for
så             | so (also: seed)
till           | to
är             | is
men            | but
ett            | a
om             | if; around, about
hade           | had
de             | they, these/those
av             | of
icke           | not, no
mig            | me
du             | you
henne          | her
då             | then, when
sin            | his
nu             | now
har            | have
inte           | inte någon = no one
hans           | his
honom          | him
skulle         | 'sake'
hennes         | her
där            | there
min            | my
man            | one (pronoun)
ej             | nor
vid            | at, by, on (also: vast)
kunde          | could
något          | some etc
från           | from, off
ut             | out
när            | when
efter          | after, behind
upp            | up
vi             | we
dem            | them
vara           | be
vad            | what
över           | over
än             | than
dig            | you
kan            | can
sina           | his
här            | here
ha             | have
mot            | towards
alla           | all
under          | under (also: wonder)
någon          | some etc
eller          | or (else)
allt           | all
mycket         | much
sedan          | since
ju             | why
denna          | this/that
själv          | myself, yourself etc
detta          | this/that
åt             | to
utan           | without
varit          | was
hur            | how
ingen          | no
mitt           | my
ni             | you
bli            | to be, become
blev           | from bli
oss            | us
din            | thy
dessa          | these/those
några          | some etc
deras          | their
blir           | from bli
mina           | my
samma          | (the) same
vilken         | who, that
er             | you, your
sådan          | such a
vår            | our
blivit         | from bli
dess           | its
inom           | within
mellan         | between
sådant         | such a
#.........这里部分代码省略.........

german_stopwords = parse_stop_word(u'''
|source: http://snowball.tartarus.org/algorithms/german/stop.txt
aber           |  but

alle           |  all
allem
allen
aller
alles

als            |  than, as
also           |  so
am             |  an + dem
an             |  at

ander          |  other
andere
anderem
anderen
anderer
anderes
anderm
andern
anderr
anders

auch           |  also
auf            |  on
aus            |  out of
bei            |  by
bin            |  am
bis            |  until
bist           |  art
da             |  there
damit          |  with it
dann           |  then

der            |  the
den
des
dem
die
das

daß            |  that

derselbe       |  the same
derselben
denselben
desselben
demselben
dieselbe
dieselben
dasselbe

dazu           |  to that

dein           |  thy
deine
deinem
deinen
deiner
deines

denn           |  because

derer          |  of those
dessen         |  of him

dich           |  thee
dir            |  to thee
du             |  thou

dies           |  this
diese
diesem
diesen
dieser
dieses


doch           |  (several meanings)
dort           |  (over) there


durch          |  through

ein            |  a
eine
einem
einen
einer
eines

einig          |  some
einige
einigem
einigen
einiger
einiges
#.........这里部分代码省略.........

finnish_stopwords = parse_stop_word(u'''
| source: http://snowball.tartarus.org/algorithms/finnish/stop.txt
| forms of BE

olla
olen
olet
on
olemme
olette
ovat
ole        | negative form

oli
olisi
olisit
olisin
olisimme
olisitte
olisivat
olit
olin
olimme
olitte
olivat
ollut
olleet

en         | negation
et
ei
emme
ette
eivät

|Nom   Gen    Acc    Part   Iness   Elat    Illat  Adess   Ablat   Allat   Ess    Trans
minä   minun  minut  minua  minussa minusta minuun minulla minulta minulle               | I
sinä   sinun  sinut  sinua  sinussa sinusta sinuun sinulla sinulta sinulle               | you
hän    hänen  hänet  häntä  hänessä hänestä häneen hänellä häneltä hänelle               | he she
me     meidän meidät meitä  meissä  meistä  meihin meillä  meiltä  meille                | we
te     teidän teidät teitä  teissä  teistä  teihin teillä  teiltä  teille                | you
he     heidän heidät heitä  heissä  heistä  heihin heillä  heiltä  heille                | they

tämä   tämän         tätä   tässä   tästä   tähän  tällä   tältä   tälle   tänä   täksi  | this
tuo    tuon          tuota  tuossa  tuosta  tuohon tuolla  tuolta  tuolle  tuona  tuoksi | that
se     sen           sitä   siinä   siitä   siihen sillä   siltä   sille   sinä   siksi  | it
nämä   näiden        näitä  näissä  näistä  näihin näillä  näiltä  näille  näinä  näiksi | these
nuo    noiden        noita  noissa  noista  noihin noilla  noilta  noille  noina  noiksi | those
ne     niiden        niitä  niissä  niistä  niihin niillä  niiltä  niille  niinä  niiksi | they

kuka   kenen kenet   ketä   kenessä kenestä keneen kenellä keneltä kenelle kenenä keneksi| who
ketkä  keiden ketkä  keitä  keissä  keistä  keihin keillä  keiltä  keille  keinä  keiksi | (pl)
mikä   minkä minkä   mitä   missä   mistä   mihin  millä   miltä   mille   minä   miksi  | which what
mitkä                                                                                    | (pl)

joka   jonka         jota   jossa   josta   johon  jolla   jolta   jolle   jona   joksi  | who which
jotka  joiden        joita  joissa  joista  joihin joilla  joilta  joille  joina  joiksi | (pl)

| conjunctions

että   | that
ja     | and
jos    | if
koska  | because
kuin   | than
mutta  | but
niin   | so
sekä   | and
sillä  | for
tai    | or
vaan   | but
vai    | or
vaikka | although


| prepositions

kanssa  | with
mukaan  | according to
noin    | about
poikki  | across
yli     | over, across

| other

kun    | when
niin   | so
nyt    | now
itse   | self
''')