Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in / Register
Toggle navigation
N
network-assets-reptile
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
liyang
network-assets-reptile
Commits
48c53bef
Commit
48c53bef
authored
Aug 01, 2023
by
liyang
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
feat:1.编写自由时报爬虫脚本
2.编写数据量统计脚本
parent
f0e81304
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
5 additions
and
2 deletions
+5
-2
pc_ltn.py
pc_ltn.py
+1
-1
index.py
utils/index.py
+4
-1
No files found.
pc_ltn.py
View file @
48c53bef
...
@@ -156,7 +156,7 @@ def reptile(browser=None, search_word=""):
...
@@ -156,7 +156,7 @@ def reptile(browser=None, search_word=""):
script_close
(
browser
)
script_close
(
browser
)
else
:
else
:
# 爬取数据为空
# 爬取数据为空
log
.
info
(
"
未爬取到数据
"
)
log
.
info
(
"
no reptile data
"
)
# 删除目录
# 删除目录
delete_directory
(
local_path
)
delete_directory
(
local_path
)
script_close
(
browser
)
script_close
(
browser
)
...
...
utils/index.py
View file @
48c53bef
...
@@ -127,7 +127,7 @@ def parse_ltn_time_string(time_str):
...
@@ -127,7 +127,7 @@ def parse_ltn_time_string(time_str):
number
=
int
(
time_str
.
split
(
"小時"
)[
0
])
number
=
int
(
time_str
.
split
(
"小時"
)[
0
])
time_delta
=
datetime
.
timedelta
(
hours
=
number
)
time_delta
=
datetime
.
timedelta
(
hours
=
number
)
return
int
((
datetime
.
datetime
.
now
()
+
time_delta
)
.
timestamp
())
return
int
((
datetime
.
datetime
.
now
()
+
time_delta
)
.
timestamp
())
el
se
:
el
if
"月"
in
time_str
and
"日"
in
time_str
:
try
:
try
:
datetime_str
=
time_str
.
replace
(
"月"
,
" "
)
.
replace
(
"日"
,
""
)
datetime_str
=
time_str
.
replace
(
"月"
,
" "
)
.
replace
(
"日"
,
""
)
month
,
day
=
map
(
int
,
datetime_str
.
split
())
month
,
day
=
map
(
int
,
datetime_str
.
split
())
...
@@ -136,6 +136,9 @@ def parse_ltn_time_string(time_str):
...
@@ -136,6 +136,9 @@ def parse_ltn_time_string(time_str):
return
int
(
datetime_obj
.
timestamp
())
return
int
(
datetime_obj
.
timestamp
())
except
ValueError
:
except
ValueError
:
return
None
return
None
else
:
# print(time_str)
return
datetime
.
datetime
.
strptime
(
time_str
,
'
%
Y/
%
m/
%
d'
)
# 转换 youtube 的时间
# 转换 youtube 的时间
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment