Machine studying fashions have been educated to foretell semantic details about person interfaces (UIs) to make apps extra accessible, simpler to check, and to automate. Presently, most fashions depend on datasets which can be collected and labeled by human crowd-workers, a course of that’s pricey and surprisingly error-prone for sure duties. For instance, it’s attainable to guess if a UI factor is “tappable” from a screenshot (i.e., based mostly on visible signifiers) or from doubtlessly unreliable metadata (e.g., a view hierarchy), however one approach to know for sure is to programmatically faucet the UI factor and observe the results. We constructed the Endless UI Learner, an app crawler that routinely installs actual apps from a cell app retailer and crawls them to find new and difficult coaching examples to study from. The Endless UI Learner has crawled for greater than 5,000 device-hours, performing over half 1,000,000 actions on 6,000 apps to coach three pc imaginative and prescient fashions for i) tappability prediction, ii) draggability prediction, and iii) display similarity.