Humans realize smooth communication by estimating and predicting the emotions of others and controlling and uttering their own emotions appropriately. The construction of a ``human-like'' agent that appropriately handles the emotional dynamics of self and others is an important issue in HAI. With the progress of large-scale language models in recent years, it has become possible to generate richly expressive sentences based on statistical learning, instead of the conventional stereotyped sentence generation. In addition, in order to realize human-like emotional interaction, it is desirable to refer to the brain when considering its function realization. From the above, by appropriately adding a brain organ model centered on the amygdala, which appropriately captures the dynamics of one's own and others' emotions, to a large-scale language model, it is possible to realize interactions that increase the level of emotional satisfaction of the interlocutor. Will. In this presentation, based on the functions necessary for interaction based on emotion, we will overview the level of emotion model. Next, we summarize the current state of language models and emotion research. Based on this, we propose a roadmap regarding the potential of emotion models in the amygdala function and how to fuse large-scale language models and emotion models in the future.

For a robot to assist people in home environments, it is important to handle the vocabulary of unobserved objects while learning the knowledge of places. It is assumed that there exist objects that the robot did not observe through its sensors during learning. For such a case, the robot is expected to perform household tasks on language instructions that include the vocabulary of these objects. We propose a method that integrates a large language model and a spatial concept model to enable the robot to understand language instructions that include the vocabulary of unobserved objects while learning places. Even if the objects that the user instructed the robot to search for are not included in a training dataset during learning, the number of room visits during object search can be expected to reduce by combining the inference of these models. We validated our method in an experiment in which a robot searched for unobserved objects in a simulated environment. The results showed that our proposed method could reduce the number of room visits during the search compared to the baseline method.

The creature can perceive various information intuitively from physical entities, understand their surroundings, and act adaptively. In planning for autonomous agents, utilizing affordances like living organisms to adapt to the environment and achieve goals efficiently is effective. Therefore, the purpose of this study was to extract affordance information from large-scale language models. Large-scale language models have learned knowledge from a vast amount of text written by humans and can output new text using that knowledge. Thus, it is considered that large-scale language models contain common sense and implicit knowledge that humans possess. In this study, we analyzed the output from the large-scale language model GPT-3 and constructed a knowledge network by extracting knowledge from it. The experiments showed that using this knowledge network enables the acquisition of affordances similar to those of humans.

Transfer learning is a powerful technique that allows a model trained on one task to be fine-tuned on a different but related task. In this presentation, we will explore how to use transfer learning to perform text classification using the BERT model and it's variaty from HuggingFace. BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained model that has been shown to achieve state-of-the-art results on a wide range of natural language understanding tasks. By fine-tuning BERT on a labeled dataset of text classification, we can quickly and easily train a high-performance model with minimal data and computational resources. We will demonstrate how to fine-tune BERT using the Hugging Face library and provide tips and best practices for getting the most out of this powerful technique. Attendees will leave with a solid understanding of how to use transfer learning for text classification and the knowledge to implement their own text classification models using BERT. We will also show how to rapidly implement this with open source MLflow and Transformers.

In the big data era, a good Machine learning model requires massive training data with labels. When there is less data available in a target domain, Cross-domain recommendations are useful to leverage richer data from a source domain to improve performance of the recommendation. Cross-domain recommendation has gained lots of interest in recent years. In this talk, we will talk about the overview of CDR, what are the existing CDR approaches, demonstrate a hands-on application for user profile prediction using CDR, as well as the challenges and future directions.

As the development of deep learning models are getting more and more popular, for industries, building good deep learning models case by case is still very expensive. At the meantime, with more and more models are being published to the public through different platforms, transfer learning which is used for transferring knowledge (i.e., the feature encoding) from the pre- trained model to a new model is becoming a technology for practical usage. This work introduces an approach to use a pre- trained deep learning model to compute features for downstream models. As an application of this approach, it demonstrate how to use transfer learning to improve training performance for image clustering by combining transfer learning for vectorization of image data and k-means for image clustering. To demonstrate the effect, we used the tf_flowers dataset, and ResNet50 for featurization of images.

EdTech, which utilizes educational data, is being implemented in society in the United States and other countries. In Japan, a roadmap for the utilization of educational data has been formulated. The EdTech widely ranges from already implemented to emerging. We will explore ELSI based on an analytical framework of three categories: legal normative fundamental principles, legal normative regulation, and cultural backgrounds.

The GIGA School Project has accelerated the use of ICT in education. While there are great expectations to be gained from the utilization of learning data, there are also great ethical and legal concerns. As experts in ELSI, we have organized ELSI based on the classification of EdTech into non-AI and AI systems. In this manuscript, we categorize EdTech into (1) digital teaching materials, (2) classroom support and learning support, and (3) learning management and school administration support, and organize the mapping between each basic technology and ELSI.

This paper aims to explore the issues that require consideration in discussions surrounding Ethical, Legal, and Social Implications (ELSI) of digitizing education including the datatification of education, the utilization of educational data, EdTech, and AI in Education (AIED). The discussions will encompass the handling of personal information and the treatment of learners based on the data obtained. However, these issues are not exhaustive. To elaborate, we will identify ELSI issues pertaining to EdTech and AIED from the perspectives of Ethical, Legal, and Social implications, and evaluate the relationships and differences between these three perspectives. Specific issues such as advocacy, the right to free-ride, the right to education, and social acceptance will also be examined. Finally, based on the results of the social acceptance survey, the conditions necessary for the social adoption of EdTech and AIED are identified.

Japan has lagged behind other developed countries in the use of ICT at schools for a while. However, in the past couple of years, the opportunities of the use of ICT in schools are rapidly increasing with the development of educational policies such as educational responses to new coronavirus infections and “personalized learning”. In the midst of these situations, ELSI's response to the further development of ICT utilization and EdTech will be an urgent issue. In the U.S., a leading EdTech country, the utilization of learning data in schools is being promoted. On the other hand, a number of the ELSI cases, including the appropriateness and reliability of the data use or personal data transfer without permission, have been emergent in the country. There were problematic cases of data transfer in public schools to the army recruiters and the polices. Especially, the polices made a list of children who were supposed to be committed the crimes based on academic achievement at schools. To avoid those cases, stricter ELSI protections have been constructed in the federal act and regulations, as well as the voluntary guidelines developed by non-governmental organizations. The reporters will show the expected ethical, legal, and social issues of the EdTech policy in Japan from the perspective and experience of cases in the U.S.

Program tracing is an effective method to overcome ambiguous understanding in programming education. To provide explicit program tracing tasks, a learning support system has been developed. Positive results were demonstrated in evaluation experiments using this system. However, the system only handles pre-registered questions and feedback by the creator, making it difficult to be used in a more general context. To address this, the authors developed a system that can automatically generate problems and feedback by analyzing arbitrary programs, enabling learners to think about errors in their own programs and conduct debugging exercises. This paper describes the evaluation experiments using the developed system and its results, which revealed areas requiring improvement in future development.

Refinement activities that bring code closer to better are important in programming learning.We have developed a code-sharing platform with a mechanism for sharing only code that is close in level. The authors believe that sharing codes that are close to the learners' strategy is beneficial in refinement activities, and have proposed a method to evaluate the strategy of codes and a method to calculate the closeness of strategy as similarity.In this paper, we implemented a filtering method that displays only codes that exceed a certain level of similarity to the ranking, and conducted an experiment to compare the ranking of the proposed method with that of the conventional method, targeting expert programmers. The results suggest that learning by using the ranking of the proposed method encourages code refinement activities.

We are developing a system that automatically collects the history of students' program editing, compilation, and execution history during programming exercises, recognizes the history and its temporal changes, and detects impasses in the exercises. Using the results, we have also developed a system to support individual advice for students who are stuck in a programming exercise. In this study, we use machine learning to detect whether a student is still stuck or not after the next 10 minutes from 20 minutes of automatically collected exercise activity history. We employed several machine learning methods to identify the impasse using the training data from the exercise history collected in previous exercises. Although the dataset is relatively small, an F value of 0.95 was obtained as the impasse detection rate by learning with a decision tree.

In 2021, the national center for university entrance examinations developed the code block programming and the data utilization modules compliant with the portable custom interaction specification formulated by the 1EdTech consortium. These modules can be used to create and present programming and data usage questions in computer-based testing. We demonstrate the effectiveness of the modules through experiments.

The authors have proposed a knowledge structure for arithmetic sentences and have attempted to estimate the learner's comprehension state based on this structure. The construction of a learner model that represents the learner's comprehension state is an important element in the study of Intelligent Tutoring System. machine learning methods such as Knowledge Tracing describe the learner's state transitions using probabilistic models, while the problem exploration process is not explicitly not explicitly described. In contrast, this study defines problem making as a search of the problem space based on knowledge structures and expresses these transition rules by the degree to which they take into account the constraints of the knowledge structures. This is a new understanding state estimation model that combines a semantic description of search with a quantitative update formula for the degree of consideration. This paper reports the implemented search model and its simulated results based on real logs.

This study addresses interpretable methods for the learner response prediction task that uses the training data of the question text for each question and its responses to predict the response to new question-learner pairs.

Computerized adaptive testing tends to select and present items frequently with high discrimination parameters. This tendency leads to bias of item exposure. To address this shortcoming, we propose difficulty constrained uniform adaptive testing. During the first stage, the method selects and presents the optimal item from a uniform item group generated by a state-of-the-art uniform test assembly technique. In the second stage, the method selects and presents the optimal item with a difficulty parameter value within the neighborhood of the examinee's ability estimate from the whole item pool. Numerical experiments results underscore the effectiveness of the proposed method.

In learning, learners sometimes make mistakes on the same problem repeatedly and get stuck. Helping stalled learners with auxiliary problems can be effective. Auxiliary problems are problems that help the learner understand the original problem. The learner who is presented with an auxiliary problem can also notice errors in the original problem while solving the problem. The authors have been working on the automatic generation of auxiliary problems for mechanics. Specifically, we have studied ``how to generate problems with consistent deletion'' based on the causal inference theory of force and motion by Mizoguchi et al. and created rules for the automatic generation of auxiliary problems. In this paper, we implement the rules in a system and develop a system that can generate auxiliary problems automatically.

Question generation (QG) for reading comprehension, a technology for automatically generating questions related to given reading passages, has been used for educational purposes. Recently, QG methods based on deep neural networks have succeeded in generating fluent questions that are pertinent to given reading passages. However, conventional methods focus only on generating questions and cannot generate answers to them. Furthermore, they ignore the relation between question difficulty and learner proficiency, making it hard to determine an appropriate difficulty for each learner. To resolve these problems, we propose a new method for generating question–answer pairs that considers their difficulty, estimated using item response theory. The proposed difficulty controllable generation is realized by extending two pre-trained transformer models, namely, BERT and GPT-2. Experimental results show that our method can generate fluent question-answer pairs with arbitrary difficult.

It is important to understand the mental state of the learner in the teaching/learning process, but sometimes the learner intentionally does not express negative emotions. In this study, intentionally not expressing such negative emotions is called emotional concealment, and we examined the possibility of estimating it from the learner's biometric information. In particular, we focused on the feature points of the learner's face, and attempted to clarify the relationship between microexpressions and emotion concealment. Using the results, we quantified emotion concealment from time-series data of biometric information, and tried to detect it as an abnormal value. In addition, we examined the development of feedback generation that tells the teacher that the teacher is hiding emotions using the results.

Sentence compression focusing on user queries is useful for presenting the results of Web search. Previous methods construct a sentence compression model using training data consisting of source sentences, queries, maximum lengths of compressed sentences and compressed sentences, but it is costly to create such data, and in practice they only use pseudo-created training data. In this paper, we propose a sentence compression method that does not require such training data. The proposed method generates candidate compressed sentences based on dependency structures, and selects compressed sentences that satisfy grammatical and semantic constraints. Since the candidate compressed sentences are selected under the constraint that they contain queries, query-focused sentence compression can be achieved.

In recent years, chatbots have come to be used in a variety of settings. However, current chatbots generally respond in a uniform manner, and are not able to respond in a diverse manner like humans. One method to solve this problem is to add characterization to chatbots' speech. Although there have been attempts to add characterization to chatbot speech, they do not necessarily reflect the characteristics of actual character speech, and it has not been fully clarified which elements of speech contribute to the development of characterization. Therefore, in this study, we extracted stylistic features of character utterances of "Uma Musume Pretty Derby," which is a popular sales title in the character business. In order to extract features, we compared the speech of the characters with that of ordinary women, targeting frequently occurring words, parts of speech, topics, and sentence-final expressions, in order to extract speech that is unique to the characters. As a result, it became clear that stylistic features corresponding to the setting of each character were found in personal expressions, emotional verbs, and sentence-final expressions with particles and auxiliary verbs.

Achieving native-speaker proficiency is a formidable task for language learners. What can be done to draw closer to a native-speaker’s own sense of language? The present project attempted to provide one answer to that question through the creation of an L1 vocabulary 3D map, referred to below as L1 Map, which shows the relationships between words in visual form. A set of 185 words was drawn from the elementary school-level English textbook Let’s Try! 1. Each word in the set was associated with added-word vectors with 300 dimensions from the pretrained model for English, fastText, a library for the learning of word embeddings. To make L1 Map useable as a visual tool for learning, the 300 dimensions in these word vectors were reduced to three dimensions using UMAP (Uniform Manifold Approximation and Projection for Dimension Reduction). K-means clustering analysis was performed on the data with three dimensions. The results indicate that L1 Map may be a useful tool for learners of English. Additionally, L1 Map turns fastText’s inability to distinguish between multiple meanings into a strength, offering visualizations of word origins.

Emotions are important in analyzing human conversations. The current response selection methods, however, do not handle emotions in response selection. This paper proposes a model that simultaneously learns emotions expressed with speakers' multi-turn utterances and their responses using predicted emotion sequence. Evaluation using the MELD dataset showed that our model achieved higher accuracies in predicting emotions and responses compared with the methods that learn response selections without emotion analysis do.

Recently, instruction tuning has been attracting significant attention as a method for training generalizable language models (e.g., ChatGPT). Although various prompts have been manually created for instruction tuning, it has not been clarified what kind of prompts are optimal for obtaining cross-task generalization ability. This study presents \emph{instruction optimization}, which optimizes training prompts by leveraging bilevel optimization, and we clarify what kind of prompts are optimal for instruction tuning. Experimental results demonstrate that instruction optimization enhances the diversity of prompts and improves the generalization performance in a zero-shot setting, whereas using the same examples rather than a variety of exemplars is more effective in a few-shot setting.

It is known that individual behavior patterns are heterogeneous temporally with regard to various types of human behavior. The distributions of Inter-Event Times (IET) of such event time series show a long-tail distribution like a power-law distribution. In particular, it has been reported that there are individual differences in the shape of the IET distributions of physical action event time series (e.g., arm movements) and smartphone touch event time series, but what the differences in the IET distributions mean has not been clarified. We focused on the click event time series of Web services and confirmed that the statistical pattern including the IET distribution for each user changes over time. The results of our analysis suggest that this change in the statistical pattern can be interpreted as habituation to the operation of the Web service. Our results are important in that they provide insight into the origin of temporal heterogeneity common to various human behaviors. Furthermore, our results could be utilized for the improvement of the user experience of web services.

In recent years, trouble on social networking services has become an issue. One of the causes is "agitation. However, "agitation" in SNS has not been defined in detail, and accurate detection has not been achieved. In this study, we classify agitation expressions in order to achieve accurate detection of "agitation" in SNS. As target data, we collected tweets that were judged by the analyst to be agitation using the Twitter API, extracted and annotated the means, intentions, and topics considered to be constituent elements. The χ-square test and factor analysis were conducted using the statistical data of the components included in each tweet. In the factor analysis, factors such as promotion, judgment, slander, mounting, and inducement were obtained. The obtained results show the characteristics of the components of "agitation" and their relationships in SNS, and are considered to be useful for classifying agitated tweets.

The purpose of this research is to enable long-term forecasting of time series data with multiple seasonal variations with a low amount of calculation and high accuracy. We use FEDformer (Zhou et al., 2022) as a baseline model. Since FEDformer performs attention in the frequency domain, it is possible to capture the periodicity even when there are multiple seasonal variations. In addition, it is designed to lighten the calculation by sampling frequency components when performing matrix calculation in the frequency domain. However, because Zhou et al. neglected an important condition in this sampling, the reduction of computational cost is small. We demonstate that, by sampling the frequency component based on the amplitude, it is possible to maintain accuracy with a small number of samples. As a result, in the long-term forecasting of time series data with multiple seasonal variations, we achieved higher accuracy than other models with less computational cost.

With the development of quantum technology, the application of quantum computer to machine learning has gained attention. Tensor network, one of quantum-inspired algorithms has also been applied to machine learning and successfully solves various kinds of tasks. In this work, with the aim of evaluating their applicability for real world problems, we employ quantum neural network, one of quantum machine learning algorithms, and matrix product state, a well studied tensor network to predict Japanese stock returns. Based on their predictions, we further conduct the investment simulation and measure the performance, comparing with benchmarks, linear regression and classical neural network models. Our experimental result shows the matrix product state model outperforms other models. On the other hand, while performances of quantum and classical neural network models vary depending on the market conditions, the quantum model has the better performance than the classical one in the latest market environment.

Anomaly Communication Detection is important to ensure the safety of industrial control systems (ICS). However, it is difficult to create detection rules for all the various communication protocols used in an ICS, including proprietary ones. Therefore, anomaly communication detection using Bidirectional Encoder Representations for Transformers (BERT) for feature extraction of packet payload has attracted attention, since it learns the characteristics of packet payloads without prior knowledge and can handle a wide range of protocols. In this paper, we conduct experiments to investigate the features and usefulness of this method. Specifically, we (1) measure the detection performance of random rewriting of payloads of typical protocols and (2) confirm the performance improvement by applying an overdetection correction technique. Through these experiments, we demonstrate the performance of anomaly communication detection using BERT for feature extraction of packet payloads and consider its effectiveness.

We address semi-supervised continual learning for learning with labeled data, which is assumed to be abundant in existing continual learning research, in situations where it is not sufficiently available in the real world. We propose a semi-supervised continual learning method that uses soft outputs of a deep neural network as pseudo-labels for an image classification task, and define a semi-supervised continual learning scenario. In the experiments, the proposed and compared methods are used in several continual learning scenarios and evaluated based on the step-by-step accuracy and final accuracy as the new class increases, as well as the average accuracy over all steps.

Sign language production (SLP) aims to generate sign language frame sequences from the corresponding spoken language text sentences. Existing approaches to SLP either rely on autoregressive models that generate the target sign frames sequentially, suffering from error accumulation and high inference latency, or non-autoregressive models that attempt to accelerate the process by producing all frames parallelly, which results in the loss of generation quality. To optimize the trade-off between speed and quality, we propose a semi-autoregressive model for sign language production (named SATSLP), which maintains the autoregressive property on a global scale but generates sign pose frames parallelly on a local scale, thus combining the best of both methods. Furthermore, we reproduced the back-translation transformer model, in which a spatial-temporal graphical skeletal structure is encoded to translate to text for evaluation. Results on the PHOENIX14T dataset show that SATSLP outperformed the baseline autoregressive model in terms of speed and quality.

PINN is a PDE solver realized as a neural network by incorporating the PDEs to be satisfied into the network as physical constraints. In this study, focusing on how to select the collocation points, we propose an active learning method to improve the efficiency of PINN learning. The proposed method uses variational inference based on dropout learning to evaluate the uncertainty of the solution estimate by PINN and defines an acquisition function for active learning based on the uncertainty. Then, by probabilistically sampling collocation points using the acquisition function, a reasonable solution can be obtained faster than random sampling. We demonstrate the effectiveness of the method using Burgers’ equation and the convection equation. We also show experimentally that the choice of the collocation points can affect the loss function, the fitting of initial and boundary conditions, and the sensible balance of PDE constraints.

It is important for infrastructure managers to maintain a high standard to ensure user satisfaction during a lifecycle of infrastructures. Surveillance cameras and visual inspections have enabled progress toward automating the detection of anomalous features and assessing the occurrence of the deterioration. Frequently, collecting damage data constraints time consuming and repeated inspections. One-class damage detection approach has a merit that only the normal images enables us to optimize the parameters. Simultaneously, the visual explanation using the heat map enable us to understand the localized anomalous feature. We propose a civil-purpose application to automate one-class damage detection using the fully-convolutional data description (FCDD). We also visualize the explanation of the damage feature using the up-sampling-based activation map with the Gaussian up-sampling from the receptive field of the fully convolutional network (FCN). We demonstrate it in experimental studies: concrete damage and steel corrosion and mention its usefulness and future works.

Sleep quality can be affected by several factors, such as sleep environment, lifestyles and so on. Existing sleep quality evaluation methods did not consider the impact of these factors. This research proposed a novel deep learning architecture with multiple-factors for sound-based sleep quality assessment. Utilizing sleep sound for sleep quality evaluation is low-cost and contactless, also, sound data can reflect several physical behaviors such as snore, cough and body movements, which are important when human experts manually evaluate sleep quality. This research utilized VAE-LSTM to learn sleep patterns in sleep sound and applied Gated Variable Selection Network (GVSN) to select useful information in factors. We recorded whole night sleep sounds of more than 100 subjects by microphone at home and collected questionnaires for experiment. The results show that the proposed method can perform accurate sleep quality prediction as well as factor importance analysis.

Real-time detection and tracking of fish in a school might have multifold functions to contribute to collective behavior analysis and precision fish farming practices like individual monitoring for growth, anomaly detection, population counting, feed management, and guided and directional control for various measurements. Although the method called SORT (simple online and real-time tracking) and its extensions (e.g., SORT, OC-SORT) are widely used for tracking humans (e.g., pedestrians), such algorithms might have a great challenge to track objects with a similar appearance (e.g., animals). Among them, tracking fish in a school is much more difficult because of their similarity in size, shape, and appearance. Therefore, this research aims to compare the performance of multiple object tracking (MOT) methods, specifically SORT and its latest extension, OC-SORT, to find suitable algorithms for further applied research on various physical and behavioral measurements of fish for individual, collective behavioral analysis, and precision fish farming practices in the future.

A key challenge for e-commerce platforms is how to build trust between buyers and sellers and help buyers make better purchasing decisions. In this regard, researchers are interested in addressing the information asymmetry between buyers and sellers. In this study, we focus on featured products that are often sold at a higher price than the original price and examine whether signals hidden in the seller's presentation of such products can mitigate this information asymmetry. To do so, we compute a sentiment score for each product presentation text based on word frequency through text analysis. Finally, we drop this sentiment score into a logistic regression model to see if these variables can significantly influence buyers' purchase intentions as signals. In conclusion, we find that the calculated sentiment scores can have a significant impact on customers' purchase intentions and can be regarded as a new signal to reduce information asymmetry between buyers and sellers.

We propose a knowledge-aware attentional neural network (KANN) for dealing with recommendation tasks by extracting knowledge entities from user reviews and capturing understandable interactions between users and items at the knowledge level. The proposed KANN can not only capture the inner attention among user (item) reviews but also compute the outer attention values between users and items before generating corresponding latent vector representations. These characteristics enable the explicit preferences of users for items to be learned and understood. Furthermore, our results and analyses highlight the relatively high effectiveness and reliability of KANN for explainable recommendation. Our code is publicly released at https://github.com/liuyuncoder/KANN.

Financial data contains a vast amount of information relating to financial terms and business entities. Without properly organizing the data, it is difficult for artificial intelligence (AI) systems to utilize financial knowledge effectively and efficiently. In this paper, we, therefore, present our ongoing work on constructing a Japanese Financial Knowledge Graph to aid the financial domain AI applications.

The use of general personality traits, specifically the Big-Five personality traits, in recommendation systems has been widely explored and adopted in various fields such as music, film, and literature. However, research on personality-aware recommendations in specific domains, such as finance and education, where domain-specific psychological traits such as risk tolerance and behavioral biases play a crucial role in explaining user behavior, remains limited. To bridge this gap, this study investigates the effectiveness of personality-aware recommendations in financial stock recommendation tasks. Firstly, the paper demonstrates the utility of general personality traits in financial stock recommendations. Secondly, this paper shows that incorporating domain-specific psychological traits along with general personality traits enhances the performance of the recommendation system. Thirdly, we propose a personalized stock recommendation model that incorporates both general personality traits and domain-specific psychological traits as well as interaction data, resulting in superior performance compared to baseline models.

Recommending attractive products is generally implemented in internet shopping to increase customers’ purchasing desire. When conducting such sales promotion measures, rather than treating all customers equally, a higher degree of effectiveness can be expected when grouping customers by industry sector and their purchasing behavior features and then implementing sales promotions tailored to each group. Furthermore, customizing the promotion for each customer will improve the effectiveness of the measures even further. The method proposed in this study first extracts a core group of customers for each industry sector and clusters other customers according to their distance from the core group in a multidimensional space of dozens of purchase behavior feature variables such as purchase amount, frequency, and the number of categories. Collaborative filtering is then performed within each customer cluster to recommend products suitable for each customer. We confirmed that the proposed method improves LTV by recommending specialized products to customers in clusters with low LTV.

Sarcasm detection is a challenging task, which identifies expressions that have the opposite meaning of what is written. Most previous works only measure sentiment polarity in sentences. However, more features are needed for improving the result. In this paper, we intend to compare different feature extraction methods including n-gram, sentiment, punctuation, and part of speech features for sarcasm detection. Firstly, sarcastic data are collected using Twitter API, and preprocessed by removing all the hashtags, mentions and URLs. Then, after all features were extracted, they are combined by One Hot Encoding. Finally, we use two classification methods: Support Vector Machine and Logistic Regression for comparison. In our experimental results, n-gram feature gives the best performance compared to the other individual features. Support Vector Machine gives a better performance than logistic regression with an F1-measure of 79.64%. This shows the potential of combining different features for sarcasm detection.

Representing text into a multidimensional space can be done with sentence embedding models such as Sentence-BERT (SBERT). However, training these models when the data has a complex multilevel structure requires individually trained class-specific models, which increases time and computing costs. We propose a two step approach which enables us to map sentences according to their hierarchical memberships and polarity. At first we teach the upper level sentence space through an AdaCos loss function and then finetune with a novel loss function mainly based on the cosine similarity of intra-level pairs. We apply this method to three different datasets: two weakly supervised Big Five personality dataset obtained from English and Japanese Twitter data and the benchmark MNLI dataset. We show that our single model approach performs better than multiple class-specific classification models.

Answering questions involving financial documents using a language model requires the ability to recognize tabular and textual data, as well as numerical reasoning. This article explains the challenges, recent progress, and our approach to tackling this problem by incorporating external structured knowledge. We also introduce our financial knowledge graph (KG) linking companies to people, industries, and facts extracted from public financial filings. The KG is part of our work to advance machine-learning models for more complex financial questions beyond the scope of the previous models and datasets.

Recognizing financial terminologies from text is essential for key information retrieval and content understanding. In general, financial terminologies do not appear in single-token form but are composed of several tokens. Also, in terminologies, a proper name might have diverse expressions, like abbreviations and morphological inflection, which sacrifice the recognition performance on recall. In this paper, along with transformer-based language models, i.e. XLM-Roberta, we propose a mechanism to train the neural classifier to distinguish terminologies from plain text, by learning from the sequential tags of targeting tokens. Initially, the targeting tokens are from a list of terminologies. To involve the diverse expressions, we inventively generate different morphologies of terminologies and utilize them to extend the targeting tokens. The experiments' results prove that this mechanism shows a convincing improvement in identifying financial terms from plain text.

In this paper, I demonstrate that a reasonably sized set of handcrafted features (866, applied to titles and description texts separately) plus encoded metadata can be used to predict the click-through rates of the dynamic Responsive Search Ad format, exceeding the performance of some fine-tuned Transformer-based large language models at a fraction of the training cost.

The magic square has been pursued as a theme of recreational mathematics since ancient times. Most of existing methods are rule-based methods, so only a few special magic squares can be generated. This paper proposes a multi-stage evolutionary strategy that hierarchically classifies the constraints of magic squares into three stages of rows, columns, and diagonals, and uses the individuals satisfying each layer's constraints as the initial individuals for the next layer. A magic square is a state in which the sum of each element in a square matrix in rows, columns, and diagonals is equal to a constant value. A square matrix that satisfies only the constraints in rows and columns is called a semi-magic square. In this research, the semi-magic square was divided into two layers, and by adding a lower layer that generates individuals satisfying only the row constraints, the need for exploration in the row direction was eliminated, resulting in a significant reduction in the number of searches. Experiments compared the execution time and generation number between the proposed method and previous study, and achieved a significant reduction of the execution time.

Machine learning of characteristic structures from tree structured data is studied intensively. We propose a method for acquiring characteristic tag tree patterns with wildcards from positive and negative tree structured data, by using tree edit distance and evolutionary learning. We report preliminary experimental results on our evolutionary learning method.

Deep learning-based image classification models are vulnerable to adversarial examples (AEs). Existing defense methods have improved the classification accuracy for AEs, but the classification accuracy for clean images without perturbations decreases. To solve this problem, we propose a new defense mechanism called self-examination mechanism. In the proposed method, the input image is first classified. Then, the inference process of the classification model is verified using SHapley Additive exPlanations (SHAP), a method of explainable AI. If the input image is abnormal, the classification is performed again based on the output of SHAP. Thus, misclassification of AEs can be prevented without significantly reducing the classification accuracy of clean images. Evaluations on ResNet and WideResNet trained with CIFAR10 demonstrate that our method improves the accuracy for AEs and hardly reduces the accuracy for clean images.

Advances in measurement technology have made it possible to acquire various data during a match, and advanced data analysis is being used to plan team tactics, and evaluate and coach players. Analysis of invasive sports such as soccer is difficult because the game situation is continuous in time and space, and multiple agents individually recognize the game situation and make decisions. In the previous study using deep reinforcement learning, which is one of the representative agent modelings, they have often considered the team as one agent and evaluated the players and teams who hold the ball in each discrete event. Therefore it was difficult to evaluate the behavior of multiple players, including players far from the ball, in a spatio-temporally continuous state space. In this study, based on a deep reinforcement learning model with a discrete action space in a continuous state space that mimics Google Research Football (a reinforcement learning platform for soccer), the actions in actual games are evaluated by estimating the action-value function of multiple players. In the experiment, the calculated player evaluation index was verified using the data of one season of a team in the J-League.