Why is fasttext so fast?

 Posted on Wed May 19 2021  |  1 minutes  |  156 words  |  近藤綾乃

Features of fasttext Improved objective function Consideration of negative samples This should not affect training time. Change in optimization method Use of stochastic optimization If it affects the learning time, it should be this one Implementation in C language This is the most effective, isn’t it? If we implement it in ## pytorch, it won’t be much different from word2vec. It would depend on the amount of data to be trained. [Read More]
natural language processing  distributed representation 

Creating data in Natural Language Inference (NLI) format for Sentence transformer

 Posted on Wed Feb 17 2021  |  2 minutes  |  376 words  |  近藤綾乃

Using the Sentence Transformer to I’m trying to use Sentence Transformer to infer causal relationships between documents. If we can do this, we can extract the cause and symptoms of the incident from the report. So, I wondered if NLI could be used for feature learning to extract causal information. I thought. What is NLI? Inference of the relationship between two sentences Forward Inverse Unrelated The three relations are. Apply to causal relationships If we apply the three relationships of NLI to causality, the following patterns are possible. [Read More]
NLI  sentence transformer  technology  natural language processing  document classification 

How to train a Japanese model with Sentence transformer to get a distributed representation of a sentence

 Posted on Wed Feb 3 2021  |  3 minutes  |  508 words  |  近藤綾乃

. BERT is a model that can be powerfully applied to natural language processing tasks. However, it does not do a good job of capturing sentence-wise features. Some claim that sentence features appear in [ CLS\ ], but This paper](https://arxiv.org/abs/1908.10084) claims that it does not contain that much useful information for the task. Sentence BERT is a model that extends BERT to be able to obtain features per sentence. The following are the steps to create Sentence BERT in Japanese. [Read More]
technical  natural language processing  BERT  distributed representation  technology  sentence transformer 

Using BART (sentence summary model) with hugging face

 Posted on Tue Jan 19 2021  |  2 minutes  |  345 words  |  近藤綾乃

BART is a model for document summarization Derived from the same transformer as BERT Unlike BERT, it has an encoder-decoder structure This is because it is intended for sentence generation This page shows the steps to run a tutorial on BART. Procedure install transformers Run ``sh pip install transformers Run summary 2. Run the summary from transformers import BartTokenizer, BartForConditionalGeneration, BartConfig model = BartForConditionalGeneration.from_pretrained('facebook/bart-large') tokenizer = BartTokenizer.from_pretrained('facebook/bart-large') ARTICLE_TO_SUMMARIZE = "My friends are cool but they eat too many carbs. [Read More]
engineering  natural language processing  python  technology  distributed representation  sentence generation 

Procedure for obtaining a distributed representation of a Japanese sentence using a trained Universal Sentence Encoder

 Posted on Mon Jun 22 2020  |  2 minutes  |  405 words  |  近藤綾乃

. A vector of documents can be obtained using Universal Sentence Encoder. Features Supports multiple languages. Japanese is supported. Can handle Japanese sentences as vectors. Usage Clustering, similarity calculation, feature extraction. Usage Execute the following command as preparation. pip install tensorflow tensorflow_hub tensorflow_text numpy Trained models are available. See the python description below for details on how to use it. import tensorflow_hub as hub import tensorflow_text import numpy as np # for avoiding error import ssl ssl. [Read More]
technology  natural language processing  universal sentence encoder  technology  python  distributed representation 

Enumerating Applications of Document Classification Problems Only

 Posted on Thu Jun 18 2020  |  3 minutes  |  550 words  |  近藤綾乃

Applying the Document Classification Problem You’ve learned about machine learning, but you don’t know how to use it! Isn’t it? It is easy to overlook this if you don’t pay attention to it when you study it, but if you don’t keep your antennas up, you won’t know how to use it. If you don’t keep your antennae up, you won’t know how to use it. Since a tool is only a tool if it is used, you should make a note of how you use your newly acquired tool. [Read More]
technical  natural language processing  technology  document classification 

A note on how to use BERT learned from Japanese Wikipedia, now available

 Posted on Wed Jun 17 2020  |  1 minutes  |  472 words  |  近藤綾乃

huggingface has released a Japanese model for BERT. The Japanese model is included in transformers. However, I stumbled over a few things before I could get it to actually work in a Mac environment, so I’ll leave a note. Preliminaries: Installing mecab The morphological analysis engine, mecab, is required to use BERT’s Japanese model. The tokenizer will probably ask for mecab. This time, we will use homebrew to install Mecab [Read More]
technical  natural language processing  bert  technology  python  distributed representation 

How to use NeuralClassifier, a library that provides a crazy number of models for document classification problems

 Posted on Mon Jun 15 2020  |  3 minutes  |  537 words  |  近藤綾乃

[! [](https://1.bp.blogspot.com/-YlMb8v77MN4/XurdQSzS1yI/AAAAAAAAg6Y/oSZrJ0c9yxYbzQnNNTynRvZnEp-xGE7NwCK4BGAsYHg/s320/AFE90C8A-A49C- 4475-9F05-50E2D56D5B63.jpeg)](https://1.bp.blogspot.com/-YlMb8v77MN4/XurdQSzS1yI/AAAAAAAAg6Y/oSZrJ0c9yxYbzQnNNTynRvZnEp-xGE7NwCK4 BGAsYHg/s1920/AFE90C8A-A49C-4475-9F05-50E2D56D5B63.jpeg) NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit is a python library for multi-label document classification problems published by Tencent. For more information, see [NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit](https://github.com/Tencent/NeuralNLP- NeuralClassifier) NeuralClassifier is designed for quick implementation of neural models for hierarchical multi-label classification task, which is more challenging and common in real-world scenarios. for more details. NeuralClassifier is designed for quick implementation of neural models for hierarchical multi-label classification task, which is more challenging and common in real-world scenarios. [Read More]
technical  natural language processing  neural classifier  technology  python  fasttext 

I even did a document classification problem with Fasttext

 Posted on Sat Jun 13 2020  |  2 minutes  |  801 words  |  近藤綾乃

Summary of what I’ve done with Fasttext to the document classification problem. Facebook research has published a document classification library using Fasttext. Fasttext is easy to install in a python environment. Run time is fast. Preliminaries I decided to tackle the task of document classification, and initially thought. NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit. However, it was not [Read More]
technical  natural language processing  fasttext  technology  distributed representation  document classification 

I made a summary text generation AI for making short-form news

 Posted on Sat Jun 13 2020  |  6 minutes  |  1078 words  |  近藤綾乃

. We have successfully trained a model to automatically generate titles from news texts using a machine translation model based on deep learning. Preliminaries In the past, I was involved in a project to automatically generate titles from manuscripts for online news. In the past, I was involved in a project to automatically generate titles from manuscripts for online news. In order to tackle this project, I was looking into existing methods. [Read More]
gpu  engineering  sentence shortening  technology  sentence generation  deep learning  natural language processing