This study evaluates the LSTM code generated by seven assistants ChatGPT 4.5, GitHub Copilot, Deepseek 3, Perplexity, Gemini 2.0 Pro, Claude 3.7 Sonnet, and Meta's Llama from a single standardized prompt, on three indices.
Abstract
Generative AI coding assistants are increasingly used to write machine-learning code, yet their ability to produce reliable LSTM implementations for financial prediction remains underexplored. This study evaluates the LSTM code generated by seven assistants ChatGPT 4.5, GitHub Copilot, Deepseek 3, Perplexity, Gemini 2.0 Pro, Claude 3.7 Sonnet, and Meta’s Llama from a single standardized prompt, on three indices (Nikkei 225, S&P 500, STOXX Europe 600). Each assistant’s generated script was re-executed over independent runs; accuracy (MAE, MSE, RMSE, R2, execution time) is reported as mean ± standard deviation on the original price scale, complemented by a static code-quality analysis (Pylint, Radon, SonarQube, Pytest, Bandit). The assistants converge on nearly identical LSTM architectures, so performance differences arise mainly from data-handling and code-correctness defects: Meta’s Llama near-zero errors are an artifact of normalized-scale metrics combined with a shuffled train/test split (data leakage), and once corrected its accuracy is among the weakest; Gemini 2.0 Pro, once its predictions are evaluated consistently on the price scale, is among the most accurate assistants. Differences are validated with Diebold–Mariano and Wilcoxon tests. AI-generated forecasting code can be accurate but is not uniformly trustworthy: its generated preprocessing and evaluation code must be audited before use.
It is concluded that while LLMs hold genuine promise within AI trading systems, robust deployment requires careful task decomposition, rigorous backtesting protocols, and domain-aware fine-tuning strategies.
: Deep learning (DL) is now routine in software defect prediction (SDP), yet how much it improves on traditional machine learning (ML), how stable that improvement is, and what governs it remain contested. We synthesized 45 empirical studies published between 2015 and 2024, comprising 1540 performance estimates, using Hedges’ g of the area underx the receiver operating characteristic curve (AUC) and a random-effects model. The pooled effect was g = 0.61 (95% CI 0.53 – 0.70; p < 0.001), but heterogeneity was substantial ( I ² = 87.7%; τ ² = 0.076) and the 95% prediction interval was 0.07 – 1.16. The lower prediction limit is close to the null and is therefore more decision-relevant than the positive mean alone: a new setting may show little practical gain. Hybrid architectures produced the largest subgroup estimate ( g = 0.90), whereas the gain under cross-project defect prediction was less than half that under within-pro-ject defect prediction (0.31 vs. 0.67). A mild publication-year association explained only about 15% of between-study heterogeneity, and no sample-size association was detected. Publication-bias and leave-one-out diagnostics did not identify a single dominant study, but these diagnostics do not eliminate bias from primary-study design or baseline tuning. The evidence therefore supports a conditional, not universal, DL advantage whose practical value depends on architecture, validation protocol, data distribution, tuning quality, and deployment cost.
Wei-Xiang Gan, Jia-Lin Liu, Mengfei Xiao et al.· Engineering and Computing In...· 0 citations
Deadline pressure in software development often drives coding shortcuts, leading to internal quality degradation known as code smells. These structural anomalies contribute to technical debt accumulation and complicate system maintenance over time. This study develops an automated classification model to detect code smell contamination in the Python backend ecosystem. The methodology uses the Random Forest ensemble algorithm integrated with the Synthetic Minority Over-sampling Technique (SMOTE) for class balancing. Data mining on GitHub with high-reputation criteria extracted 137,728 code samples from 7 large-scale repositories using the Radon multi-metric tool. To simulate human error in real-world scenarios, 5% random noise was inserted into the labeling data. Testing using the confusion matrix shows the proposed model achieves highly stable and balanced performance, with average precision, recall, and f1-score of 0.95 in both macro and weighted averages. Ablation study analysis proves that SMOTE intervention effectively maintains detection consistency in minority class categories. Feature importance ranking identifies the Logical Lines of Code (LLOC) metric as the most crucial indicator with 37.65% influence weight, followed by LOC and Blank metrics. This research provides an automated quality assurance system for developers to detect code refactoring opportunities at an early stage.
Treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail is supported.
Nhat Thanh Vu, M. Rashid, Fariza Sabrina· Electronics· 0 citations
Early-stage startup success is notoriously difficult to predict, yet the stakes for getting it right - whether for investors, accelerators, or founders themselves - are substantial. Most existing machine learning approaches lean heavily on generic features from Crunchbase or PitchBook, and in doing so tend to miss a category of arguably more informative signals: the domain-specific structural properties of elite accelerator programs like Y Combinator (YC). This paper introduces a YC-Inspired Feature Engineering (YIFE) framework that incorporates batch cohort timing, founder technical depth (proxied via public GitHub activity), team composition, industry category, and geographic cluster alongside conventional funding and operational variables. We evaluate five classifiers - Logistic Regression, Random Forest, XGBoost, Support Vector Machine, and a Multilayer Perceptron - on a curated dataset of 4,323 YC-funded companies spanning 2005-2024 using a temporal train-test design. XGBoost with YIFE achieves the strongest performance, with an F1-score of 0.85 and an area under the receiver operating characteristic curve of 0.91 on the held-out W21-S24 cohort, representing a 19-percentage-point F1 improvement over a generic Crunchbase baseline (0.85 vs. 0.66) and an 8-percentage-point improvement over a replicated literature baseline (0.85 vs. 0.77). SHAP (SHapley Additive exPlanations)-based interpretation identifies funding-related variables, batch-year context, and team size among the most influential model features, while prior FAANG (Facebook (now Meta), Amazon, Apple, Netflix, and Google) experience contributes comparatively little within the available founder-profile sample. Because several funding-related predictors may only be observable after the initial accelerator stage and overlap conceptually with the outcome definition, the framework is best interpreted as a domain-contextualized retrospective classification approach for the YC ecosystem rather than a strict ex-ante forecasting tool. These findings suggest the potential value of accelerator-specific feature engineering while motivating future validation under fixed prediction horizons and across non-YC accelerator ecosystems. The analytical code and pipeline are made publicly available to support this and other future work in computational entrepreneurship and venture analytics.
Siddharth Gupta, Pratham Namdev, Shubham Nagar et al.· Cureus Journal of Computer S...· 0 citations
From the ways agents exploited their harness--reading sibling runs through shared git state, leaving notes to"future runs"in persistent memory--the authors distill five design rules for evaluating autonomous agents.
N. Askarbekuly, Mohamad Al Mdfaa, Ahmed Helaly et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
