크롤링 [JAVA, JSP]

환경 : eclipse , hichart, codepen

• codepen

CodePen

• hichart

Interactive javascript charts library

• jsoup jar 파일 다운로드

 

• 크롤링하기 위한 웹페이지 → 소스보기 클릭
• 크롤링을 위한 예제

/*
		  <div class="box-contents">
                        <a href="/movies/detail-view/?midx=85813">
                            <strong class="title">범죄도시 2</strong>
                        </a>

                        <div class="score">
                            <strong class="percent">예매율<span>44.2%</span></strong>
		  
		*/

• Mainclass

해당 jsoup connect객체를 생성 → 크롤링 하고자하는 url 선택

public static void main(String[] args) throws Exception {
		Document doc = Jsoup.connect("<http://www.cgv.co.kr/movies/?lt=1&ft=0>").get();
	}

• 접근 방법

Elements titles = doc.select("div.box-contents strong.title");

Elements percents = doc.select("div.box-contents div.score strong.percent span");

• 출력 - 제목, 예매율

for(int i=0; i<19; i++) {
			Element title = titles.get(i);
			System.out.println(title.text());
			Element percent = percents.get(i);
			System.out.println(percent.text());
		}

• MovieChart

public class MovieChart {
	
	public static List getCGVdata() throws Exception {
		Document doc = Jsoup.connect("<http://www.cgv.co.kr/movies/?lt=1&ft=0>").get();
		
		Elements titles = doc.select("div.box-contents strong.title");
		
		Elements percents = doc.select("div.box-contents div.score strong.percent span");
		
		List list = new ArrayList();
		
		for(int i=0; i<19; i++) {
			Element title = titles.get(i);
			Element percent = percents.get(i);
			System.out.println(title.text() + " " + percent.text());
			
			String t = title.text();
			// 44.2% -> 44.2 만가지고오고싶다
			double p =Double.parseDouble(percent.text().replace("%", ""));
			
			list.add(new MovieVo( t,p ));
		}		
		return list;
	}

}

• Movievo → getter setter
• index.jsp

<%@page import="MovieDatas.MovieChart"%>
<%@page import="dto.MovieVo"%>
<%@page import="java.util.List"%>
<%@ page language="java" contentType="text/html; charset=UTF-8"
    pageEncoding="UTF-8"%>
    
    
<%
	List list = MovieChart.getCGVdata();
	for(MovieVo v : list){
		System.out.println(v.toString());
	}
	
	// list -> json
	String jsonData = "[";
	for(MovieVo v : list){
		jsonData += "{ name:'" + v.getTitle() + "', y:" + v.getPercent() + "}," ;
	}
	jsonData = jsonData.substring(0, jsonData.lastIndexOf(","));
	
	jsonData += "]";
	
	System.out.println(jsonData);
	
	// 여기서 담으면 js에서 활용할 수 있다. 키값으로 통해서 
	request.setAttribute("jsonData", jsonData);

%>

 

Pie charts are very popular for showing a compact overview of a composition or comparison. While they can be harder to read than column charts, they remain a popular choice for small datasets.

Highcharts.chart('container', {
	  chart: {
	    plotBackgroundColor: null,
	    plotBorderWidth: null,
	    plotShadow: false,
	    type: 'pie'
	  },
	  title: {
	    text: '2022.05.27 movie chart'
	  },
	  tooltip: {
	    pointFormat: '{series.name}: <b>{point.percentage:.1f}%</b>'
	  },
	  accessibility: {
	    point: {
	      valueSuffix: '%'
	    }
	  },
	  plotOptions: {
	    pie: {
	      allowPointSelect: true,
	      cursor: 'pointer',
	      dataLabels: {
	        enabled: true,
	        format: '<b>{point.name}</b>: {point.percentage:.1f} %'
	      }
	    }
	  },
	  series: [{
	    name: 'Brands',
	    colorByPoint: true,
	    data: <%=request.getAttribute("jsonData")%>
	  }]
	});