接上一节( )。
先上图
幻猿·简易爬虫主界面
关于JavaFx
JavaFx是在2007年5月的JavaOne大会上公之于众的，而第一个正式版本v1.0是在2008年12月份才发布的。JavaFX技术主要应用于创建RIA(Rich Internet Application，富网络应用)应用。
依赖引入
<!-- JavaFx -->
<dependency>
<groupId>de.roskenet</groupId>
<artifactId>springboot-javafx-support</artifactId>
<version>${springboot-javafx.version}</version>
</dependency>
<dependency>
<groupId>org.greenrobot</groupId>
<artifactId>eventbus</artifactId>
<version>3.1.1</version>
</dependency>
<dependency>
<groupId>org.controlsfx</groupId>
<artifactId>controlsfx</artifactId>
<version>8.40.16</version>
</dependency>
- springboot-javafx-support库的作用是使JavaFx支持SpringBootd。
- eventbus看名称就知道是个事件订阅库，在后续交互界面多的时候，切换界面可以通关订阅事件的方式来处理界面之间的数据交互。
- controlsfx则是一个JavaFx的组件库，组件很丰富，我们这里用到了StatusBar组件。
核心代码
这一节是基于前边的版本调整的，引入了界面逻辑，相对之前的版本有了很大的改变，这里就罗列一些主要的类及方法，完整的项目在后边的会给出Github地址。
SpringBoot
SpringBoot的启动类调整有点大，首先要继承AbstractJavaFxApplicationSupport，其次调整main方法如下：
public static void main(String[] args) {
launch(SpiderApplication.class, DashBoardView.class, new CustomSplash(), args);
}
switchView方法：这个方法主要作用是界面切换。
public static void switchView(Class<? extends AbstractFxmlView> from, Class<? extends AbstractFxmlView> to, Object object) {
try {
logger.debug("从 {} 跳转到 {}", from, to);
StopWatch started = StopWatch.createStarted();
AbstractFxmlView fromViewer = BeanManager.getBean(from);
AbstractFxmlView toViewer = BeanManager.getBean(to);
if (!bus.isRegistered(fromViewer.getPresenter()) && hasSubscribe(fromViewer.getPresenter())) {
bus.register(fromViewer.getPresenter());
logger.info("registered:{}", fromViewer.getPresenter().getClass());
}
if (!bus.isRegistered(toViewer.getPresenter()) && hasSubscribe(toViewer.getPresenter())) {
bus.register(toViewer.getPresenter());
logger.info("registered:{}", toViewer.getPresenter().getClass());
}
if (bus.isRegistered(fromViewer.getPresenter())) {
logger.debug("发布隐藏事件");
bus.post(new ViewEvent(ViewEvent.ViewEvenType.hide, fromViewer, fromViewer.getPresenter()));
}
Platform.runLater(() -> {
Abstract Java FxApplicationSupport.showView(to);
if (bus.isRegistered(toViewer.getPresenter())) {
logger.debug("发布显示事件");
bus.post(new ViewEvent(ViewEvent.ViewEvenType.show, toViewer, toViewer.getPresenter()));
}
if (object != null) {
logger.debug("跳转参数:{}", object);
bus.post(object);
}
logger.debug("跳转页面耗时:{}", started.getTime());
});
} catch (Exception e) {
logger.error("跳转页面异常", e);
}
}
JavaFx
JavaFx部分主要由2部分组成，一个是界面元素Fxml以及FxmlView，一个则是对界面元素的控制逻辑Controller。主界面核心代码如下。
Fxml和FxmlView
<?xml version="1.0" encoding="UTF-8"?>
<?import javafx.geometry.*?>
<?import javafx.scene.control.*?>
<?import javafx.scene.layout.*?>
<?import org.controlsfx.control.*?>
<AnchorPane prefHeight="641.0" prefWidth="1027.0" xmlns="#34; xmlns:fx="#34; fx:controller="mobi.huanyuan.spider.ui.controller.DashBoardController">
<BorderPane prefHeight="640.0" prefWidth="1026.0">
< top >
<MenuBar prefHeight="25.0" prefWidth="1027.0" AnchorPane.leftAnchor="0.0" AnchorPane.rightAnchor="0.0" AnchorPane.topAnchor="0.0">
<Menu text="文件">
<MenuItem fx:id="setting" text="配置" />
<MenuItem fx:id="exit" text="退出" />
</Menu>
<Menu text="关于">
<MenuItem fx:id="about" text="关于" />
</Menu>
</MenuBar>
</top>
<left>
<TreeView fx:id="treeView" onMouseClicked="#treeViewClick" prefHeight="590.0" prefWidth="226.0" BorderPane.alignment="TOP_LEFT" />
</left>
<center>
<Pane prefHeight="590.0" prefWidth="899.0" BorderPane.alignment="CENTER">
<HBox alignment="CENTER_RIGHT" maxHeight="-Infinity" maxWidth="-Infinity" prefHeight="490.0" prefWidth="350.0">
<VBox alignment="TOP_RIGHT" maxHeight="-Infinity" maxWidth="-Infinity" prefHeight="599" prefWidth="449">
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="网址">
< padding >
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox. margin >
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="关键字">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="爬取深度">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="爬取网址进程数">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="分析数据进程数">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="存储数据进程数">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="存储类型">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
<Label alignment="CENTER_RIGHT" maxWidth="200.0" prefHeight="40.0" text="本地地址">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</Label>
</VBox>
<VBox maxHeight="-Infinity" maxWidth="-Infinity" prefHeight="599" prefWidth="449">
<TextField fx:id="url" maxWidth="200" prefHeight="40.0" promptText="网址">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</TextField>
<TextField fx:id="keys" maxWidth="200" prefHeight="40.0" promptText="关键字">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</TextField>
<TextField fx:id="maxDepth" maxWidth="200" prefHeight="40.0" promptText="2">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</TextField>
<TextField fx:id="htmlThreadNum" maxWidth="200" prefHeight="40.0" promptText="2">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</TextField>
<TextField fx:id="parseThreadNum" maxWidth="200" prefHeight="40.0" promptText="2">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</TextField>
<TextField fx:id="storeThreadNum" maxWidth="200" prefHeight="40.0" promptText="2">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</TextField>
<ComboBox fx:id="storeType" maxWidth="200.0" prefHeight="40.0" promptText="--存储类型--">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</ComboBox>
<TextField fx:id="localPath" maxWidth="200" prefHeight="40.0" promptText="数据存储地址">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<VBox.margin>
<Insets bottom="10.0" left="5.0" right="5.0" top="10.0" />
</VBox.margin>
</TextField>
</VBox>
</HBox>
<HBox alignment="CENTER" layoutY="480.0" maxHeight="-Infinity" maxWidth="-Infinity" prefHeight="100.0" prefWidth="350.0">
<Button fx:id="startBtn" onAction="#start" text="开始">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<HBox.margin>
<Insets bottom="10.0" left="5.0" right="10.0" top="10.0" />
</HBox.margin>
</Button>
<Button fx:id="stopBtn" onAction="#stop" text="结束">
<padding>
<Insets bottom="5.0" left="5.0" right="5.0" top="5.0" />
</padding>
<HBox.margin>
<Insets bottom="10.0" left="10.0" right="5.0" top="10.0" />
</HBox.margin>
</Button>
</HBox>
</Pane>
</center>
<bottom>
<StatusBar fx:id="statusBar" />
</bottom>
</BorderPane>
</AnchorPane>
@FXMLView(value = "/fxml/DashBoard.fxml", title = "幻猿·简易爬虫")
public class DashBoardView extends AbstractFxmlView {
}
Controller
@FXMLController
public class DashBoardController extends BaseController implements Initializable {
private static Logger logger = LoggerFactory.getLogger(DashBoardController.class);
private Image rootIcon;
private Image dayIcon;
private Image keyWordIcon;
private Image demoIcon;
@FXML
private MenuItem exit;
@FXML
private MenuItem setting;
@FXML
private MenuItem about;
@FXML
private TreeView<String> treeView;
@FXML
private TextField url;
@FXML
private TextField keys;
@FXML
private TextField maxDepth;
@FXML
private TextField htmlThreadNum;
@FXML
private TextField parseThreadNum;
@FXML
private TextField storeThreadNum;
@FXML
private ComboBox<StoreType> storeType;
@FXML
private TextField localPath;
@FXML
private Button startBtn;
@FXML
private Button stopBtn;
@FXML
private StatusBar statusBar;
@Autowired
private SettingMapper settingMapper;
@Autowired
private SpiderHistoryMapper spiderHistoryMapper;
@Autowired
private Spider spider;
@Override
public void initialize(URL location, ResourceBundle resources) {
rootIcon = new Image(this.getClass().getResourceAsStream("/images/history.png"), 25, 25, false , false);
dayIcon = new Image(this.getClass().getResourceAsStream("/images/date.png"), 25, 25, false, false);
keyWordIcon = new Image(this.getClass().getResourceAsStream("/images/keyword.png"), 25, 25, false, false);
demoIcon = new Image(this.getClass().getResourceAsStream("/images/demo.png"), 25, 25, false, false);
initMenus();
ObservableList<StoreType> storeValues = FXCollections.observableArrayList(StoreType.values());
storeType.getItems().addAll(storeValues);
storeType.getSelectionModel().select(StoreType.MYSQL);
Setting setting = settingMapper.selectByPrimaryKey(Constants.SettingDefaultId);
final DirectoryChooser file Chooser = new DirectoryChooser();
String workDir = System.getProperties().getProperty("user.dir");
String storeLocalPath = StringUtils.isBlank(setting.getLocalPath()) ? workDir : setting.getLocalPath();
localPath.setText(storeLocalPath);
localPath.setOnMouseClicked(event -> {
configureFileChooser(fileChooser, storeLocalPath);
File file = fileChooser.showDialog(localPath.getParent().getScene().getWindow());
if (file != null) {
logger.info("localPath: {}", file);
localPath.setText(file.getAbsolutePath());
}
});
ChangeListener<String> numberValidListener = (observable, oldValue, newValue) -> {
if (!newValue.matches("d*")) {
maxDepth.setText(newValue.replaceAll("[^d]", ""));
}
};
maxDepth.textProperty().addListener(numberValidListener);
htmlThreadNum.textProperty().addListener(numberValidListener);
parseThreadNum.textProperty().addListener(numberValidListener);
storeThreadNum.textProperty().addListener(numberValidListener);
initTreeView();
}
private void initMenus() {
exit.setOnAction(actionEvent -> Platform.exit());
setting.setOnAction(event -> {
SpiderApplication.showView(SettingView.class, Modality.WINDOW_MODAL);
});
about.setOnAction(event -> {
Dialog<?> dialog = new Dialog<>();
dialog.setTitle("关于幻猿·简易爬虫");
dialog.setContentText("nt一个简易的爬虫系统。nn" +
"t基于SpringBoot2、MyBatis、JavaFx技术实现。nn" +
"tttttttversion: 0.0.1nn");
dialog.getDialogPane().getButtonTypes().add(ButtonType.CLOSE);
Node closeButton = dialog.getDialogPane().lookupButton(ButtonType.CLOSE);
closeButton.managedProperty().bind(closeButton.visibleProperty());
closeButton.setVisible(false);
dialog.showAndWait();
});
}
private static void configureFileChooser(final DirectoryChooser fileChooser, String defaultPath) {
fileChooser.setTitle("选择文件夹");
if (StringUtils.isNotBlank(defaultPath)) {
File file = new File(defaultPath);
if (file.exists()) {
fileChooser.setInitialDirectory(file);
}
}
}
//====================================================================================
// Tree
//====================================================================================
/**
* 设置TreeView
*/
public void initTreeView() {
TreeItem<String> root = new TreeItem<>("近30天记录", new ImageView(rootIcon));
root.setExpanded(true);
treeView.setRoot(root);
SpiderHistoryExample example = new SpiderHistoryExample();
SpiderHistoryExample.Criteria criteria = example.createCriteria();
LocalDate thirtyDaysAgo = LocalDate.now().minusDays(30);
criteria.andDayGreaterThanOrEqualTo( Integer .parseInt(DateTimeFormatter.BASIC_ISO_DATE.format(thirtyDaysAgo)));
example.setOrderByClause("DAY DESC");
List<SpiderHistory> historyList = spiderHistoryMapper.selectByExample(example);
for (SpiderHistory history : historyList) {
TreeItem<String> keyWordsNode = new TreeItem<>(history.getKeyWords(), new ImageView(keyWordIcon));
boolean found = false;
for (TreeItem<String> dayNode : root.getChildren()) {
if (dayNode.getValue().contentEquals("" + history.getDay())) {
dayNode.getChildren().add(keyWordsNode);
found = true;
break;
}
}
if (!found) {
TreeItem<String> dayNode = new TreeItem<>(
"" + history.getDay(),
new ImageView(dayIcon)
);
root.getChildren().add(dayNode);
dayNode.getChildren().add(keyWordsNode);
}
}
TreeItem<String> day = new TreeItem<>("Demo", new ImageView(demoIcon));
Arrays.asList("Java", "Python", "JavaScript", "JavaFx", "SpringBoot").forEach(s -> {
TreeItem<String> node = new TreeItem<>(s, new ImageView(keyWordIcon));
day.getChildren().add(node);
});
root.getChildren().add(day);
}
/**
* TreeView 点击事件
*/
public void treeViewClick() {
TreeItem<String> selectedItem = treeView.getSelectionModel().getSelectedItem();
if (null != selectedItem && selectedItem.isLeaf()) {
fillData(selectedItem.getValue(), selectedItem.getParent().getValue());
}
}
/**
* 填充数据
*/
private void fillData(String keyword, String day) {
String maxDepth1 = "2";
String htmlThreadNum1 = "2";
String parseThreadNum1 = "2";
String storeThreadNum1 = "2";
String storeLocalPath1 = System.getProperties().getProperty("user.dir");
String storeType1 = StoreType.MYSQL.getType();
String url1 = "#34;;
if (!"Demo".equals(day)) {
SpiderHistoryExample example = new SpiderHistoryExample();
SpiderHistoryExample.Criteria criteria = example.createCriteria();
criteria.andDayEqualTo(Integer.parseInt(day)).andKeyWordsEqualTo(keyword);
SpiderHistory history = spiderHistoryMapper.selectByExample(example).get(0);
if (null != history) {
maxDepth1 = "" + history.getMaxDepth();
htmlThreadNum1 = "" + history.getHtmlThreadNum();
parseThreadNum1 = "" + history.getParseThreadNum();
storeThreadNum1 = "" + history.getStoreThreadNum();
storeLocalPath1 = history.getStoreLocalPath();
storeType1 = history.getStoreType();
url1 = history.getUrl();
}
}
url.setText(url1);
keys.setText(keyword);
maxDepth.setText(maxDepth1);
htmlThreadNum.setText(htmlThreadNum1);
parseThreadNum.setText(parseThreadNum1);
storeThreadNum.setText(storeThreadNum1);
storeType.getSelectionModel().select(StoreType.valueOf(storeType1));
localPath.setText(storeLocalPath1);
}
//====================================================================================
// StatusBar
//====================================================================================
private void startTask() {
Task<Void> task = new Task<Void>() {
@Override
protected Void call() throws Exception {
while (!Spider.isStopping) {
Thread.sleep(200);
updateMessage(String.format("已爬取页面:%d | 待爬取页面:%d | 待分析页面:%d | 待存储页面:%d",
SpiderQueue.getUrlSetSize(), SpiderQueue.getUnVisitedSize(),
SpiderQueue.waitingMineSize(), SpiderQueue.getStoreSize()));
}
done();
return null;
}
};
statusBar.textProperty().bind(task.messageProperty());
statusBar.progressProperty().bind(task.progressProperty());
// remove bindings again
task.setOnSucceeded(event -> {
statusBar.textProperty().unbind();
statusBar.progressProperty().unbind();
});
new Thread(task).start();
}
//====================================================================================
// Button
//====================================================================================
public void start() {
SpiderHtmlConfig spiderHtmlConfig = SpiderHtmlConfig.builder()
.keys(Arrays.asList(StringUtils.replace(keys.getText(), "，", ",").split(",")))
.maxDepth(Integer.parseInt(maxDepth.getText()))
.minerHtmlThreadNum(Integer.parseInt(htmlThreadNum.getText()))
.minerParseThreadNum(Integer.parseInt(parseThreadNum.getText()))
.minerStoreThreadNum(Integer.parseInt(storeThreadNum.getText()))
.storeType(storeType.getSelectionModel().getSelectedItem())
.storeLocalPath(localPath.getText())
.build();
// Spider spider = BeanManager.getBean(Spider.class);
spider.start(spiderHtmlConfig, url.getText());
startBtn.setDisable(true);
// start statusBar
startTask();
}
public void stop() {
stopBtn.setDisable(true);
// Spider spider = BeanManager.getBean(Spider.class);
spider.stop();
startBtn.setDisable(false);
stopBtn.setDisable(false);
// stop statusBar
statusBar.textProperty().unbind();
statusBar.progressProperty().unbind();
statusBar.setProgress(0);
}
}
关于我
程序界的老猿，自媒体界的新宠 じ☆ve
程序界的老猿，自媒体界的新宠 じ☆ve
联系方式：1405368512@qq.com
