前端错误日志上报及网站统计(sentry+matomo)

本文配合 rancher1.6(手头一个测试集群没升级到最新的 2.x)讲解如何搭建并配置日志错误上报框架Sentry及网站统计分析框架matomo 的搭建及接入 vue(本文以iview-admin为例)项目。

背景简述

  • sentry 项目运行过程中,难免出现 bug,前端不像后端可以很方便的采集项目日志(比如 log4j + elk),导致每次出问题后还原车祸现场费时费力。另外现在随着 vue 等兴起,构建项目时打成 min.js,无疑又加大了前端定位问题的难度。而 sentry 是一款专注于错误采集的框架,支持常见的主流语言image.png采集,聚合,并推送错误信息。注意,sentry 并不是日志平台(e.g. log4j + elk),也不是监控平台,sentry 专注于项目中的 Error 信息的采集,聚合,报警。
  • matomo 前身 Pwiki,是一款开源的 web 网站分析利器。类似于 Google Analytics。具体的特性,参见 Premium Web Analytics ,比如绘制页面热力图,录制会话,访问漏斗,A/B Test 等(这几样都是收费插件功能)。

注意:本文假设你已经有 rancher1.6 的环境

安装

matomo

rancher 创建 matomo

在 rancher 主机上

## 创建必要文件夹
mkdir -p /data/matomo/{config,logs,php,maxmind}/

## 安装maxmind ip数据库
wget -P /tmp/ https://github.com/maxmind/geoipupdate/releases/download/v4.0.3/geoipupdate_4.0.3_linux_amd64.deb
dpkg -i /tmp/geoipupdate_4.0.3_linux_amd64.deb
mv /etc/GeoIP.conf{,.bak}
cat << EOF | sudo tee -a /etc/GeoIP.conf
AccountID 0
LicenseKey 000000000000
EditionIDs GeoLite2-Country GeoLite2-City GeoLite2-ASN
DatabaseDirectory /data/matomo/maxmind
EOF

## 下载最新maxmind数据库
geoipupdate
ls -lah /data/matomo/maxmind/
总用量 67M
drwxr-xr-x 2 root root 4.0K 7月 7 17:25 .
drwxr-xr-x 6 root root 4.0K 7月 7 17:23 ..
-rw------- 1 root root 0 7月 7 17:25 .geoipupdate.lock
-rw-r--r-- 1 root root 6.3M 7月 7 17:25 GeoLite2-ASN.mmdb
-rw-r--r-- 1 root root 57M 7月 7 17:25 GeoLite2-City.mmdb
-rw-r--r-- 1 root root 3.7M 7月 7 17:25 GeoLite2-Country.mmdb

## 定时更新ip数据库
cat << EOF | sudo tee -a /etc/cron.d/geoipupdate
50 2 * * 4 root /usr/bin/geoipupdate
EOF

## 设置php.ini
cat << EOF | sudo tee -a /data/matomo/php/php.ini
upload_max_filesize = 128M
post_max_size = 128M
max_execution_time = 200
memory_limit = 256M
EOF

docker-compose.yaml

version: "2"
services:
matomo:
image: matomo:latest
stdin_open: true
volumes:
- /data/matomo/config:/var/www/html/config
- /data/matomo/logs:/var/www/html/logs
- /data/matomo/php/php.ini:/usr/local/etc/php/php.ini
- /data/matomo/maxmind/GeoLite2-City.mmdb:/var/www/html/misc/GeoLite2-City.mmdb
- /data/matomo/maxmind/GeoLite2-Country.mmdb:/var/www/html/misc/GeoLite2-Country.mmdb
- /data/matomo/maxmind/GeoLite2-ASN.mmdb:/var/www/html/misc/GeoLite2-ASN.mmdb
tty: true
ports:
- 80:80/tcp
- 443:443/tcp

rancher-compose.yaml

version: "2"
services:
matomo:
scale: 1
start_on_create: true
health_check:
response_timeout: 2000
healthy_threshold: 2
port: 80
unhealthy_threshold: 3
initializing_timeout: 60000
interval: 2000
strategy: recreate
reinitializing_timeout: 60000

image.png

配置 matomo

选择中文
image.png
系统检查
image.png
配置数据库
image.png
自动建表完成
image.png
创建管理员用户(忘记截图了)
设置分析网站(可以随便创建,后边再进行修改),注意跟进实际情况修改时区
image.png
复制跟踪代码
image.png
配置 matomo
image.png
登陆
image.png

Sentry

rancher 创建 Sentry

docker-compose.yml

version: "2"
services:
cron:
image: sentry:9
environment:
SENTRY_MEMCACHED_HOST: memcached
SENTRY_REDIS_HOST: redis
SENTRY_POSTGRES_HOST: postgres
SENTRY_EMAIL_HOST: smtp
SENTRY_SECRET_KEY: SENTRY_SECRET_KEY_XXX
stdin_open: true
volumes:
- /data/sentry-data:/var/lib/sentry/files
- /data/sentry-data/config.yml:/etc/sentry/config.yml
tty: true
command:
- run
- cron
memcached:
image: memcached:1.5-alpine
web:
image: sentry:9
environment:
SENTRY_MEMCACHED_HOST: memcached
SENTRY_REDIS_HOST: redis
SENTRY_POSTGRES_HOST: postgres
SENTRY_EMAIL_HOST: smtp
SENTRY_SECRET_KEY: SENTRY_SECRET_KEY_XXX
stdin_open: true
volumes:
- /data/sentry-data:/var/lib/sentry/files
- /data/sentry-data/config.yml:/etc/sentry/config.yml
tty: true
ports:
- 9000:9000/tcp
worker:
image: sentry:9
environment:
SENTRY_MEMCACHED_HOST: memcached
SENTRY_REDIS_HOST: redis
SENTRY_POSTGRES_HOST: postgres
SENTRY_EMAIL_HOST: smtp
SENTRY_SECRET_KEY: SENTRY_SECRET_KEY_XXX
stdin_open: true
volumes:
- /data/sentry-data:/var/lib/sentry/files
- /data/sentry-data/config.yml:/etc/sentry/config.yml
tty: true
command:
- run
- worker
redis:
image: redis:3.2-alpine
postgres:
restart: unless-stopped
image: postgres:9.5
volumes:
- /data/postgresql/data:/var/lib/postgresql/data
ports:
- 5432:5432/tcp

注意把 SENTRY_SECRET_KEY 换成 sentry 的实际 secret key

rancher-compose.yml

version: "2"
services:
cron:
scale: 1
start_on_create: true
memcached:
scale: 1
start_on_create: true
web:
scale: 1
start_on_create: true
health_check:
response_timeout: 2000
healthy_threshold: 2
port: 9000
unhealthy_threshold: 3
initializing_timeout: 60000
interval: 2000
strategy: recreate
reinitializing_timeout: 60000
worker:
scale: 1
start_on_create: true
redis:
scale: 1
start_on_create: true
postgres:
scale: 1
start_on_create: true
health_check:
response_timeout: 2000
healthy_threshold: 2
port: 5432
unhealthy_threshold: 3
initializing_timeout: 60000
interval: 2000
strategy: recreate
reinitializing_timeout: 60000

先将 docker-compose.yml 保存到服务器上,用来初始化 db 和创建账号

docker-compose run --rm web upgrade
Would you like to create a user account now? [Y/n]: y
Email: anjia0532@gmail.com
Password:
Repeat for confirmation:
Should this user be a superuser? [y/N]: y
## 直到输出
Migrated:
- sentry
- sentry.nodestore
- sentry.search
- social_auth
- sentry.tagstore
- sentry_plugins.hipchat_ac
- sentry_plugins.jira_ac
Creating missing DSNs
Correcting Group.num_comments counter
## 并退出

配置 Sentry

image.png
image.png
image.png
image.png
image.png
image.png
image.png

配置 vue

本文以 iview-admin 为例

git clone https://gitee.com/anjia/iview-admin.git
cd iview-admin

sentry

注意,网上很多文档,以讹传讹的使用过时的工具,raven-js .从 5.x 后官方建议使用@sentry/browser 和@sentry/integrations

npm install --save @sentry/integrations
npm install --save @sentry/browser

修改 iview-admin\src\main.js

// The Vue build version to load with the `import` command
// (runtime-only or standalone) has been set in webpack.base.conf with an alias.
import Vue from "vue";
import App from "./App";
import router from "./router";
import store from "./store";
import iView from "iview";
import i18n from "@/locale";
import config from "@/config";
import importDirective from "@/directive";
import { directive as clickOutside } from "v-click-outside-x";
import installPlugin from "@/plugin";
import "./index.less";
import "@/assets/icons/iconfont.css";
import TreeTable from "tree-table-vue";
import VOrgTree from "v-org-tree";
import "v-org-tree/dist/v-org-tree.css";
import * as Sentry from "@sentry/browser";
import * as Integrations from "@sentry/integrations";
// 实际打包时应该不引入mock
/* eslint-disable */
if (process.env.NODE_ENV !== "production") require("@/mock");

Vue.use(iView, {
i18n: (key, value) => i18n.t(key, value),
});
Vue.use(TreeTable);
Vue.use(VOrgTree);
/**
* @description 注册admin内置插件
*/
installPlugin(Vue);
/**
* @description 生产环境关掉提示
*/
Vue.config.productionTip = false;
/**
* @description 全局注册应用配置
*/
Vue.prototype.$config = config;
/**
* 注册指令
*/
importDirective(Vue);
Vue.directive("clickOutside", clickOutside);

/* eslint-disable no-new */
new Vue({
el: "#app",
router,
i18n,
store,
render: (h) => h(App),
});

Sentry.init({
dsn: "https://xxx@xxx.xxx.com/xxx",
integrations: [
new Integrations.Vue({
Vue,
attachProps: true,
}),
],
});
npm install
npm run dev

打开 http://localhost:8080/error_store/error_store_page
分别点击两个按钮,模拟出错
image.png
打开 sentry 发现已经有错误上报了,并且对错误进行聚合。
image.png
image.png点开查看详细内容。

如果需要生成 source-map ,可以参考官方文档 https://docs.sentry.io/platforms/javascript/sourcemaps/

matomo

image.png
image.png
image.png

npm install --save vue-matomo

修改 iview-admin\src\main.js

// The Vue build version to load with the `import` command
// (runtime-only or standalone) has been set in webpack.base.conf with an alias.
import Vue from "vue";
import App from "./App";
import router from "./router";
import store from "./store";
import iView from "iview";
import i18n from "@/locale";
import config from "@/config";
import importDirective from "@/directive";
import { directive as clickOutside } from "v-click-outside-x";
import installPlugin from "@/plugin";
import "./index.less";
import "@/assets/icons/iconfont.css";
import TreeTable from "tree-table-vue";
import VOrgTree from "v-org-tree";
import "v-org-tree/dist/v-org-tree.css";
import * as Sentry from "@sentry/browser";
import * as Integrations from "@sentry/integrations";
import VueMatomo from "vue-matomo";

// 实际打包时应该不引入mock
/* eslint-disable */
if (process.env.NODE_ENV !== "production") require("@/mock");

Vue.use(iView, {
i18n: (key, value) => i18n.t(key, value),
});
Vue.use(TreeTable);
Vue.use(VOrgTree);
/**
* @description 注册admin内置插件
*/
installPlugin(Vue);
/**
* @description 生产环境关掉提示
*/
Vue.config.productionTip = false;
/**
* @description 全局注册应用配置
*/
Vue.prototype.$config = config;
/**
* 注册指令
*/
importDirective(Vue);
Vue.directive("clickOutside", clickOutside);

/* eslint-disable no-new */
new Vue({
el: "#app",
router,
i18n,
store,
render: (h) => h(App),
});

Sentry.init({
dsn: "https://xxx@xxx.xxx.com/xxx",
integrations: [
new Integrations.Vue({
Vue,
attachProps: true,
}),
],
});
Vue.use(VueMatomo, {
// Configure your matomo server and site by providing
host: "//xxxx.xxxx.com/",
siteId: xx,

// Changes the default .js and .php endpoint's filename
// Default: 'piwik'
trackerFileName: "matomo.js",

// Overrides the autogenerated tracker endpoint entirely
// Default: undefined
trackerUrl: "//xxxx.xxxx.com/matomo.php",

// Enables automatically registering pageviews on the router
router: router,

// Enables link tracking on regular links. Note that this won't
// work for routing links (ie. internal Vue router links)
// Default: true
enableLinkTracking: true,

// Require consent before sending tracking information to matomo
// Default: false
requireConsent: false,

// Whether to track the initial page view
// Default: true
trackInitialView: true,

// Whether or not to log debug information
// Default: false
debug: false,
});

// or
window._paq.push;

// or through
window.Piwik.getTracker;

打开 http://localhost:8080, 随便访问几个菜单,然后打开 matomo
image.png
image.png
路由已经有数据了
image.png
并且将用户的常规数据聚合起来

更多

其实本文只是 sentry 和 matomo 简单介绍
更深入的使用,比如 sentry,推送邮件,文中一带而过的 sourcemap,单点登录(集成内部的权限认证),自定义上报内容(将错误与用户 id 关联起来),,敏感数据脱敏等
比如 matomo, 每日发送分析报表,增加 kafka 插件,进行更深层次的挖掘,自定义上报内容(购物车等),大数据量情况下的优化,优化用户设备指纹,使用了 nginx 等反代软件后,如何正确识别真实 ip,热力图,A/B test,漏斗图等

来源:https://anjia0532.github.io/2019/07/07/sentry-and-matomo-install/

评论

此博客中的热门博文

远离你身边的煤气灯人

高可用用户中心设计

如何优雅的关闭容器

Matomo API 官方接口详情

从零开始学习iftop流量监控(找出服务器耗费流量最多的ip和端口)

kubernetes运行一个有状态的应用程序(使用StatefulSet)

ThinkPad E480 黑苹果踩坑和黑苹果常用操作备忘

免费在线接收手机短信验证码大全

20小时学习法则(The First 20 Hours: How to Learn Anything Fast)